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Description 

[0001] Tills application Is based on application Nos. 10-053206, 10-177662. and 10-279522 filed in Japan, the con- 
tents of which are hereby incorporated by reference. 

5 

BACKGROUND OF THE INVENTION 

(1) Fietd of the invention 

10 [0002] The present invention relates to a user interface in an electrical appliance and a personal computer, and 
relates to a broadcast receiving apparatus that is provided with a user interface. 

(2) Description of the Prior Art 

75 [0003] An electrical appliance such as a television set, a video cassette recorder, and an air conditioner is equipped 
with buttons, and the like for receiving user instructions on the main body or the remote controller. 
[0004] Each of the buttons typically corresponds to one function of the electrical appliance, and Is provided with 
characters or a maric that read or indicates the content of the function. The user pushes a button to have the electrical 
appliance perform a corresponding function. 

20 [0005] With the diversification of uses, multi-featured electrical appliances have been developed in order to meet 
user needs for more useful and complex functions. 

[0006] As a result, the number of buttons on the main body or the remote controller increases, leading to a bulky 
remote controller, which is inconvenient to carry. In addition, it takes much time for the user to find the button corre- 
sponding to a desired function. 

25 [0007] One proposed solution to these problems is an operation panel on a multi-featured electric appliance that is 
realized by a software. 

[0008] The operation panel is a graphical user Interface. More specifically, an electric appliance is equipped with an 

image displaying unit or the monitor on the appliance on which an image is output is used as an image displaying unit, 
and button images are displayed on the image displaying unit under the control of a software. The user designates 
30 one position on the image displaying unit using the remote controller and the like to select one of the button images. 
Here, a "button image" refers to an image that has the same function as a button. 

[0009] The operation panel displays the button images corresponding to the multiple functions so that the button 
images would be systematized in a hierarchical menus. As a result, the user may select each of the functions even if 
the number of button images displayed on the image displaying unit at one time is limited due to the resolution con- 

3$ straints. Here, a "menu" refers to all the button images displayed on the image displaying unit at one time. 

[0010] More specifically, the software that realizes the operation panel systematizes the multiple functions in a tree 
structure and displays the corresponding button images on the image displaying unit so that hierarchical menus would 
be displayed according to the tree structure. As a result, when selecting a function that is positioned at a relatively 
lower level in a tree structure, the user has to repeat the selection of a button image corresponding to a function that 

40 is positioned at a higher level than a button image corresponding to the desired function until the desired button image 
is displayed on the image displaying unit. 

[0011] Fig. 1 shows a video system 9000, which is an example of a conventional electrical appliance equipped with 
an operation panel that displays a hierarchical menus. 

[0012] The video system 9000 includes a video recording and replaying apparatus 9100, a monitor 9200, and a 
45 remote controller 9300. 

[0013] Here, the video recording and replaying apparatus 9100 has functions for recording programs, replaying re- 
corded programs, and displaying the replayed programs. The video recording and replaying apparatus 9100 detects 
signals transmitted from the remote controller 9300, performs the functions according to the signals, outputs signals 
to the monitor 9200 for displaying the operation panel on the monitor 9200, and outputs TV signals to the monitor 9200 
so for replaying and displaying recorded programs. 

[001 4] An operation panel 921 0 displayed on the image displaying unit of the monitor 9200 is realized by a software 
in the video recording and replaying apparatus 9100. and includes a cursor 9211. button images 9212 "replay", 9213 
"preset for recording", 9214 "recording", 9215 "TV", and 9216 "setting". 

[001 5] The remote controller 9300 receives the user instruction and transmits the signals representing the instruction 
ss to the video recording and replaying apparatus 9100. The remote controller 9300 includes remote control buttons 9310, 
9320, 9330, and 9340 for moving the cursor 9211 on the operation panel 9210, and a remote control button 9350 for 
determining the selection of a button image indicated by the cursor 9211. For instance, when the user pushes the 
remote control button 9330, the cursor 9211 moves down one line. 
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[001 6] Fig. 2 shows a hierarchical structure of function Items displayed on the operation panel 921 0 as button images. 
[0017] As shown in Fig. 2, the function Items are related to each other so that the function items would form a tree 
structure. For instance, the function item 'detect and replay" is a subltem of the function Item "replay' in the tree 

structure. 

s [001 8] When the user selects a button image on the operation panel 9210, the display on the operation panel 9210 
is changed according to the hierarchical structure shown in Fig. 2. 

[0019] When the user wants the video system 9000 to display a TV program, the processings described below are 
performed. 

[0020] The user pushes the remote control button 9340, with watching the operation panel 9210 as shown in Fig. 1 . 
10 In response to the user instruction, the cursor 9211 on the operation panel 9210 moves to the right and Indicates the 
button image 9215 "TV". 

[0021] Then the user pushes the remote control button 9350 to select the button image 9215. As a result, a TV 

program is displayed on the 9200. 

[0022] A conventional electrical appliance equipped with an above-described operation panel, however, raises a 
15 new problem that it is difficult for the user to easily select a desired function from a hierarchical menus. For instance, 
when the user has little knowledge of the hierarchical structure of the electrical appliance functions, it is difficult for the 
user to select a function that Is positioned at a relatively lower level of the structure. 

[0023] In addition, when an electrical appliance is equipped with combined or special functions even for the conven- 
ience of the user, it Is difficult to form an easy-to- understand hierarchical menus including these functions. In this case, 
20 even if hierarchical menus including these combined or special functions are formed, the structure of the hierarchical 
menus is complex. As a result, It is difficult for the user to be familiar with the hierarchical structure and to easily select 
a function from the menus. 

[0024] For instance, when the function structure of an electrical appliance has the highest menu Including the same 
functions as the video system 9000 (refer to Fig. 1), and when the electrical appliance is equipped with a combined 

25 function "delete Information on replayed program", it is difficult for the user to specify the menu in the hierarchical 
structure that includes the combined function. When the electrical appliance Is equipped with a special function "switch 
to a favorite program", it is difficult to include the special function in a menu in a nomial hierarchical structure. Even if 
the special function is included in a menu in a hierarchical structure, it is difficult for the user to specify the menu that 
includes the special function. 

30 [0025] Similar problem also arises for a multi-featured personal computer. 

SUMMARY OF THE INVENTION 

[0026] It is accordingly an object of the present Invention to provide a user interface apparatus that enables the user 
35 to easily designate a desired function of an electric appliance or a personal computer with high probability 

[0027] Another object of the present invention is to provide a user Interface apparatus that suggests the user in- 
structing a special function when necessary, and to provide a broadcast receiving apparatus equipped with the user 

interface apparatus. 

[0028] The above-mentioned first object may be achieved by a user interface apparatus that receives a user operation 
40 of an electrical device and notifies the electrical device of an execution instruction relating to an action according to 
the user operation, the user interface apparatus may include: a predicting unit for predicting at least one action, an 
execution of which Is supposed to be desired by a user; a user interface unit for providing a user interface in order to 
have the user designate one out of the actions that have been predicted by the predicting unit, and for receiving a 
designation of an action by the user; and a notifying unit for notifying the electrical device of an execution instruction 
45 relating to an action according to the user action designation that has been received by the user interface unit. 

[0029] A system equipped with the user interface apparatus according to the present invention predicts functional 
actions that the user would desire, so that the user easily instructs the execution of a functional action with high prob- 
ability. The system provides a user interface with which a functional action Is not necessarily searched for from hier- 
archical menus and is executed only with a designation by the user, so that the user may easily instruct the execution 
so of a functional action. Note that the prediction in this specification means to specify functional actions that are not 
directly designated by the user according to, for instance, the condition of an electrical device and information that the 
user has input. 

[0030] In addition, with the user interface apparatus according to the present invention, it is possible to recommend 
the execution of a functional action, for instance, to recommend the execution of a functional action that leads to the 
55 release of a system resource when the system resource is not sufficient. As a result, it is possible to have the user 
easily instruct the execution of a functional action that the user does not desire at present but would desire when 

recommended. 

[0031] A system equipped with the user interface apparatus, wherein the user interface unit automatically provides 
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tho user interface without being instructed by the user may predict functional actions and voluntarily provide a user 
Interface including choices of the predicted functional actions without user requirement, so that the exec ution Instruction 
of a functional action that Is not necessarily executed may be recommended. As a result, a user who is not familiar 
with the functions of the system may easily use the system. Note that the voluntarily providing of a user interface unit 

5 that the system provides a user interface not according to a user instruction but with a predetermined timing. 

[0032] The user interface apparatus that may further Include user information storage unit for storing user Information 
that Is peculiar to the user, wherein the predicting unit predicts the actions according to the user information after a 
condition of the electrical device becomes a predetemnined condition may predict actions of an electrical device ac- 
cording to the conditions peculiar to the user to recommend the optimal action of the electrical device to the user. As 

10 a result, the user may easily have the electrical device execute the action that is optimal to the user. In other words, 
the present invention provides a user interface that has not general actions for all users but actions peculiar to one 
user be designated. As a result, the second object is achieved. 

[0033] The user interface apparatus, wherein the user infomnation is taste Information on tastes of the user, and the 
predicting unit predicts actions that meet the tastes of the user according to the taste Information may predict actions 

'5 of an electrical device according to the tastes of the user to recommend the actions of the electrical device that are 
supposed to be desired by the user. As a result, when storing Information on the user tastes in the user information 
storage unit, the user may be provided with a unit of easily instructing an action that meets the user tastes. 
[0034] The user interface apparatus, wherein the user infonnation Is behavioral Information on habitual behavior of 
the user, and the predicting unit predicts actions that are necessary to keep the habitual behavior of the user according 

20 to the behavioral Information may predict actions of an electrical device according to information on the habitual be- 
havior of the user to recommend actions of an electrical device according to the habitual behavior of the user As a 
result, the user may have the electrical device execute an action that is necessary to keep the user habitual behavior 
only by designating an action, 

[0035] With the user Interface apparatus, wherein the user interface unit provides the user interface by displaying at 
25 least one image in order to have the user designate one out of the actions that have been predicted by the predicting 
unit, wherein the Image displayed in the user Interface Includes an image that shows a reason why the action has been 
predicted, a button Image in a graphical user Interface may include a reason. As a result, the user may understand the 
reason of the sudden appearance of a button image without confusion. 

[0036] The user interface apparatus, wherein the electrical device is a broadcast receiving apparatus that displays 

30 an image of a received program on a monitor, the user information storage unit stores user information that is peculiar 
to a user of the broadcast receiving apparatus, and the user interface unit provides the user interface by displaying an 
image on the monitor for having the user designate one out of the actions that have been predicted by the predicting 
unit may be used as the user Interface In a broadcast receiving apparatus. As a result, the user may easily have the 
broadcast receiving apparatus execute a recommendable action by pressing a button image and the like on the monitor 

35 in the broadcast receiving apparatus, which representing the recommendable action, via an input device such as a 
remote controller. Here, recommendable actions include channel switch to a program that meets the user tastes. 
[0037] The user interface apparatus, that may further include: a program table storage unit for storing a program 
table including information on a plurality of programs; and a user information receiving unit for receiving input of the 
user information by the user and for storing the user information in the user information storage unit, wherein the user 

40 information is program taste information that shows tastes of the user relating to programs, and the user Interface unit 
refers to the program table and the program taste information, detects a program that meets the tastes of the user 
other than a program that the broadcast receiving apparatus Is presently receiving, and displays an Image on the 
monitor for having the user designate an execution of an action of switching a received object of the broadcast receiving 
apparatus to the detected program may be equipped with a user interface with which the switch to a program that 

45 meets the tastes of the user of a broadcast receiving apparatus on a channel other than a presently selected channel 
may be executed. As a result, the user may not miss a favorite program during watching another program and may 
start watching the favorite program with an easy operation by inputting Information on the user tastes In advance. 
[0038] The user interface apparatus, that may further Include program table storage unit for storing a program table 
including information on a plurality of programs, wherein the user information is bedtime information that shows a 

50 bedtime of the user, and the user Interface unit refers to the program table and the bedtime information, compares the 
bedtime with a time when a program that the broadcast receiving apparatus is presently receiving ends, and displays 
an Image on the monitor for having the user designate an execution of an action of recording the program when the 
program ends after the bedtime may help the user to keep a bedtime, so that the user may easily keep habitual behavior 
[0039] The user interface apparatus that may further include: a general operation receiving unit for receiving a user 

55 operation of the electrical device other than the user action designation that the user interface unit receives; a general 
notifying unit for notifying the electrical device of an execution instruction according to the user operation that has been 
received by the general operation receiving unit; and a holding unit for holding execution instruction Infomiation on the 
execution Instruction according to the user operation that has been received by the general operation receiving unit, 
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wherein a the predicting unit predicts actions according to the execution instruction information that is held by the 
holding unit may receive an ordinary operation by the user via the general operation receiving unit As a result, the 
operations other than the functional actions that have been predicted to be desired by the user are operable. 
[0040] In addition, the prediction of actions by the user interface apparatus according to the execution of an electrical 

5 device action that has been instructed increases the probability that the user easily designates a desired action. As a 
result, the user interface apparatus according to the present invention may analyze an action that would be next desired 
by the user following the execution of an action, fetch the content of prediction judgement by the predicting unit, and 
improve prediction accuracy, to provide a graphical user interface including choices with which the user may easily 
instruct the execution of a functional action really desired by the user. 

10 [0041] With the user interface apparatus, wherein the predicting unit predicts actions when an execution completion 
of an action of the electrical device is detected, and the user interface unit provides the user interface when the predicting 
unit has predicted the actions, the user may generally desire to execute one functional action after the execution of a 
functional action. As a result, a provided user interface is useful with a high probability. 

[0042] With the user interface apparatus, wherein the electrical device is a broadcast program recording and replay- 
IS ing apparatus that records a plurality of broadcast programs, replays a recorded broadcast program, and displays the 
replayed broadcast program on the monitor, when it is detected that one of a series of broadcast programs has been 
replayed by the broadcast program recording and replaying apparatus, the predicting unit predicts an action of replaying 
a next broadcast program in the series of broadcast programs as one of the predicted actions, and the user interface 
unit provides the user interface by displaying at least one image for having the user designate one out of the actions 
20 that have been predicted by the predicting unit, after the replaying of a program in a series of recorded programs, the 
user may easily instruct the broadcast program recording and replaying apparatus to replay a program that follows the 
replayed program. 

[0043] With the user interface apparatus, wherein the predicting unit predicts actions when completions of all user 
operations that are necessary to have the electrical device execute one action are detected, the user may often desire 
25 to execute a functional action at the time of the completion of one user operation. As a result, the prediction of functional 
actions is useful with a high probability 

[0044] With the user Interface apparatus, wherein when an execution of the action according to the user action 
designation that has been received by the user Interface unit needs setting information according to a user operation, 
the notifying unit creates an execution instruction that includes the setting information by referring to the execution 

30 instruction information that is held by the holding unit, and notifies the electrical device of the created execution in- 
struction it may not be necessary for the user to Input again the object of functional action that has been Input. This 
means that a system that uses the user interface apparatus according to the present Invention provides a function that 
effectively uses information such as a functional action object that has been input by the user For instance, when the 
user instructs to execute a recording function, information such as the time and date of the recording and the channel 

3S of a program, and the like is input in a recording and replaying apparatus. Such input information may be used for a 
function "preset for recording of next episode", and the like. 

[0045] It is difficult to include a useful function such as the function "preset for recording of next episode" in a con- 
ventional hierarchical menu. Even if such a function is included in a hierarchical menu, it is difficult for the user to easily 
find the menu that includes the function. The present invention solves the problem and has the effect of providing the 

40 system with a function that is more useful to the user. 

[0046] With the user interface apparatus, wherein the predicting unit Includes a priority storage unit for storing priority 
information in which a priority is set for an action of the electrical device, and the predicting unit predicts actions, a 
number of which is at least one and no greater than a predetermined number, by referring to the priority information, 
the number of button images that are displayed as a user interface, i.e., the number of choices may be limited. As a 

^ result, the user may easily use the user Interface. 

[0047] With the user interface apparatus that may further include general operation receiving unit for receiving a user 
operation of the electrical device other than the user action designation that the user interface receives, wherein the 
general operation receiving unit has an operation button, which is used by the user, displays an image for assisting a 
user operation, and receives a user operation corresponding to the image via the operation button, and the user inter- 

so face unit has a designation button, which Is different from the operation button and Is used by the user, and receives 
the user action designation via the designation button, the probability that normal operability for a user who uses no 
user interface is kept may increase, and different buttons may be used for a normal operation and for designating a 
predicted functional action. As a result, the operation is easy to understood by the user. 

[0048] When the user had watched a program and temporarily switched channels during the commercials in the 
ss program, a broadcast receiving apparatus equipped with the user interface apparatus, wherein the electrical device is 
a broadcast receiving apparatus that displays an Image of a received program on a monitor, the electrical device 
includes: a receiving unit for receiving a program on a channel; and a condition detection unit for detecting a condition 
where the receiving unit had received a program on a first channel for at least a first period, the first channel was 
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switched to a second channel according to a user operation, and a second period has elapsed, the predicting unit 
predicts a switching action to the first channel as one action, an executbn of which Is supposed to be desired by the 
user when the condition is detected by the condition detection unit, the user Interface unit displays an image on a 
monitor for having the user designate an execution of the switching action to the first channel that has been predicted 

5 by the predicting unit, and receives the user action designation, and when notified of the switching action to the first 
channel by the notifying unit, the broadcast receiving apparatus controls the receiving unit and has the receiving unit 
receive the program on the first channel may prevent the user from forgetting to switch to the originally watched program, 
and the user may instantly switch to the originally watched program only by selecting an Image for switching channels. 
[0049] With the user interface apparatus that may further include: an evaluation value storage unit for storing eval- 

10 uation values that each correspond to different actions of the electrical device; a semantic relation information storage 
unit for storing semantic relation information on semantic relations between the actions of the electrical device and a 
plurality of keywords; a keyword receiving unit for receiving a designation of a keyword by the user; an Initializing unit 
for Initializing the evaluation values that are stored In the evaluation value storage unit and setting each of the evaluation 
values at a predetermined value; and an evaluation value increasing unit for referring to the semantic relation infor- 

'5 mation and for increasing an evaluation value corresponding to an action that has a semantic relation to the keyword, 
the designation of which has been received by the keyword receiving unit, wherein when the keyword receiving unit 
receives the keyword designation, the predicting unit predicts actions by specifying a predetermined number of actions 
in decreasing order of evaluation value according to the evaluation values that are stored in the evaluation value storage 
unit, and the user Interface unit provides a user interface for having the user designate one action out of the predeter- 

20 mined number of actions, which have been predicted by the predicting unit, it is possible to provide a graphical user 
interface that has a desired functional action be easily selected for a user who does not memorize a hierarchical menu 
structure and operates a system with which a variety of functional actions may be executed. 
[0050] The user interface apparatus, wherein the keyword receiving unit receives the designation of the keyword 
after creating a dedicated menu for having the user designate an action of the electrical device that includes a plurality 

25 of keywords having semantic relations to at least one of the actions of the electrical device by referring to the semantic 
relation information and displaying the dedicated menu, and the user interface unit provides the user Interface by 
displaying an Image for having the user designate one out of the predetermined number of actions may not have the 
user directly input a keyword by inputting the characters but have the user select a keyword. As a result, no burden of 
memorization is imposed on the user 

30 [0051] With the user interface apparatus that may further include: a general operation receiving unit for receiving a 
user operation of the electrical device other than the keyword designation that the keyword receiving unit receives and 
the user action designation that the user interface unit receives; and a general notifying unit for notifying the electrical 
device of an execution instruction according to the user operation that has been received by the general operation 
receiving unit, wherein the general operation receiving unit has an operation button, which is used by the user, displays 

35 an image for assisting a user operation, and receives a user operation corresponding to the image via the operation 
button, the user interface unit has a designation button, which is different from the operation button and Is used by the 
user, the keyword receiving unit receives the user keyword designation by the user via the designation button, and the 
user interface unit receives the user action designation via the designation button, the operability of normal hierarchical 
menus for a user who does not select using keywords may be kept, and different operation buttons may be used for 

40 selecting usirig keywords and for operating normal hierarchical menus. As a result, it is easy for the user to understand 
operations. 

[0052] The above-mentioned second object may be achieved by a broadcast receiving apparatus that displays an 
image of a received program on a monitor, the broadcast receiving apparatus may include: an execute control unit for 
receiving an Instruction, for controlling elements in the broadcast receiving apparatus in response to the instruction to 

<s have an action be executed; a user Information storage unit for storing user information that is peculiar to a user of the 
broadcast receiving apparatus; a user Interface unit for displaying at least one Image for having the user designate an 
action out of at least one action, an execution of which Is supposed to be desired by the user according to the user 
information that is stored in the user information storage unit, and for receiving a designation of the action by the user 
after the broadcast receiving apparatus is in a predetermined condition; and a notifying unit for notifying the execute 

50 control unit of an execution instruction relating to an action according to the user actbn designation that has been 
received by the user Interface unit. 

[0053] The broadcast receiving apparatus predicts actions that are supposed to be desired by the user according to 
the conditions peculiar to the user, and displays button images and the like on the monitor for designating a predicted 
recommendable action, so that the user may easily have the broadcast receiving apparatus execute a recommendable 
55 action by pressing a corresponding button Image and the like via the Input device such as a remote controller. As a 
result, the present invention provides a user Interface that has not general actions but recommendable actions peculiar 
to the user of the broadcast receiving apparatus be designated. As a result, the second object Is achieved. 
[0054] The broadcast receiving apparatus that may further include a program table storage unit for storing a program 
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table Including information on a plurality of progranns, wherein the user Information is program taste Information that 
shows tastes of the user relating to programs, and the user interface unit refers to the program table and the program 
taste information, detects a program that meets the tastes of the user other than a program that the broadcast receiving 
apparatus is receiving, and displays an image on the monitor for having the user designate an execution of an action 
5 of switching a received object of the broadcast receiving apparatus to the detected program may be equipped with a 
user interface with which the switch to a program that meets the tastes of the user on a channel other than a presently 
selected channel may be executed. As a result, the user may not miss a favorite program during watching another 
program and may start watching the favorite program with an easy operation. 

[0055] The user of the broadcast receiving apparatus that may further include a user information receiving unit for 
10 receiving the user information input by the user and for storing the user information in the user information storage unit 
may have the broadcast receiving device execute an action that meets the taste of the user by inputting information 
on the user tastes in advance. 

[0056] The user of the broadcast receiving apparatus that may further include a received program Information ob- 
taining unit for obtaining information on the received program by referring to the program table, and for storing the 
15 obtained information in the user information storage unit as the program taste information may not miss a program that 
is watched every week and watch another program, and may start watching the every-week-watched program with an 
easy operation. 

[0057] The broadcast receiving apparatus that may further include a program table storage unit for storing a program 
table including information on a plurality of programs, wherein the user information is bedtime information that shows 

20 a bedtime of the user, and the user interface unit refers to the program table and the bedtime information, compares 
the bedtime with a time when a presently received program ends, and displays an image on the monitor for having the 
user designate an execution of an action of recording the presently received program when the program ends after 
the bedtime may help the user to keep a bedtime, so that the user may easily keep habitual behavior. 
[0058] The broadcast receiving apparatus that may further include a temporary storage unit for temporarily storing 

25 data on the received program, wherein when notified of an execution instruction for the action of recording the received 
program by the notifying unit, the execute control unit has a recording unit in the broadcast receiving apparatus record 
the received program from a part that has been received before a notification of the execute Instruction for the actton 
of recording using the data stored in the temporary storage unit may record a program from a part that has been 
received before a notification of the execute instruction for the recording. As a result, the user may watch a recorded 

30 program with less troublesome at a future time. 

[0059] The broadcast receiving apparatus, wherein an image that the user Interface unit displays includes an Image 
that shows a reason why an action is predicted may add a reason to a button image and the like displayed on the 
monitor for having the user designate an action. As a result, the user may understand the meaning of a button image 
without confused by the sudden appearance of the button image. 

35 [0060] When the user had watched a program and temporarily switched channels during the commercials in the 
program, a broadcast receiving apparatus that displays a received Image on a monitor, the broadcast receiving appa- 
ratus may include: a receiving unit for receiving a program on a channel; a condition detection unit for detecting a 
condition in which the receiving unit had received a program on a first channel for at least a first period, the first channel 
was switched according to a user operation, and a second period has elapsed; a user Interface unit for displaying an 

40 image on the monitor in order to have a user designate an execution of a switching action to the first channel when 
the condition detection unit detects the condition, and for receiving a designation by the user; and a receptk^n control 
unit for controlling the receiving unit and for having the receiving unit receive the program on the first channel when 
the user interface un it receives the designation by the user may prevent the user from forgetting to switch to the originally 
watched program, and the user may instantly switch to the originally watched program only by selecting an image for 

45 switching channels. 

[0061] The broadcast receiving apparatus, wherein the image that the user interface unit displays includes an image 
showing that the second period has elapsed since the first channel was switched may add a reascm to a button image 
and the like displayed on the monitor for having the user designate an action. As a result, the user may understand 
the meaning of a button image without confused by the sudden appearance of the button image. 

so 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0062] These and other objects, advantages and features of the invention will become apparent from the following 
description thereof taken In conjunction with the accompanying drawings which illustrate a specific embodiment of the 
55 invention. In the Drawings: 

Fig. 1 shows a video system 9000. which is an example of a conventional electrical appliance equipped with an 
operation panel that displays hierarchical menus; 
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Fig. 2 shows the hierarchical structure including the function items that are shown on an operation panel 9210 in 
Fig. 1 as button Images; 

Fig. 3 shows a structure of a digital broadcast receiving system 1000 equipped with a user interface apparatus 

according to the first embodiment of the present invention; 

Fig. 4 is a block diagram of a controller 1110 in the digital broadcast receiving system 1000; 
Fig. 5 shows data stored in an execution instruction/apparatus condition information storage unit 1412 in the con- 
troller 1110; 

Fig. 6 shows standard menu hierarchical structure information that Is Included in standard menu display control 

information stored in the execution instruction/apparatus condition information storage unit 1412; 

Fig. 7 shows an example of data that the execution instruction/apparatus condition information storage unit 1412 

stores; 

Fig. 8 shows the data that the auxiliary menu Item information storage unit 1417 stores; 

Fig. 9 shows an example of the auxiliary menu item information stored in the auxiliary menu Item information 

storage unit 1417; 

Fig. 10 shows an example of the auxiliary menu item Information stored in the auxiliary menu item information 
storage unit 1417; 

Fig. 11 shows the change of the screen on a monitor 1200 according to user instruction; 

Fig. 12 is a flowchart showing the operation by a function execution controller 1413 related to execute control 

information; 

Fig. 1 3 shows the change of the screen on a monitor 1200 after the screen change shown in Fig. 11 ; 

Fig. 14 is a flowchart showing the process related to the display of an auxiliary menu by the controller 1110; 

Fig. 15 shows the change of the screen displayed on a monitor 1200; 

Fig. 16 shows a structure of a digital broadcast receiving system 3000 equipped with a user interface apparatus 
according to the second embodiment of the present invention; 
Fig. 17 is a functional block diagram of a controller 3110; 

Fig. 18 shows the change of an auxiliary menu displayed on a part of a monitor 1200; 

Fig. 19 shows the data that an auxiliary menu display control information storage unit 3417 stores; 

Fig. 20 is an example of the list of the numbers given to function item information 3610 and the corresponding 

function items; 

Fig. 21 is an example of the list of the numbers given to keyword information 3520 and the corresponding keywords; 

Fig. 22 is a flowchart showing the operation by an auxiliary menu controller 3418; 

Fig. 23 shows the principle of the updating operation of function item/keyword condition Information 3540; 

Fig. 24 is a flowchart showing the display keyword determination processing by an auxiliary menu controller 3418; 

Fig. 25 shows a structure of a digital broadcast receiving system 5000 equipped with a user interface apparatus 

according to the third embodiment of the present invention; 

Fig. 26 is a functional block diagram of a controller 5150; 

Fig. 27 is a flowchart showing a program switch operation responding processing, a recommend display purpose 
timer time elapse processing, and a watching determination detection purpose timer time elapse processing, which 
each are performed by a function controller 5151 ; 

Fig. 28 shows a monitor on which a recommend display appears for recommending watching the previously 
watched program; 

Fig. 29 is a functional block diagram of a controller 6150 in a digital broadcast receiving apparatus In the digital 

broadcast receiving system according to the fourth embodiment; 

Fig. 30 shows the content of user information stored in a user information storage unit 6158; 

Fig. 31 Fig. 31 is a flowchart showing the processing in a program recommendation function; 

Fig. 32 shows a monitor on which a recommend display appears for advising the user to switch channels to a 

recommendable program; 

Fig. 33 is a flowchart showing the processing in a bedtime responding function; 

Fig. 34 shows a monitor on which a recommend display appears for advising the user to record a presently watched 
program; and 

Fig. 35 shows a structure of a digital broadcast receiving system 7000 provided with a digital broadcast receiving 
apparatus 7100, which is another example of a digital broadcast receiving apparatus according to the fourth em- 
bodiment. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0063] The embodiments of the present Invention will be described below with reference to figures. 
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(First Embodiment) 

[0064] A user interface apparatus according to the first embodiment of the present invention will be described with 
reference with Figs. 3 to 15. 

5 

(Structure) 

[0065] Fig. 3 shows a structure of a digital broadcast receiving system 1 000 equipped with a user Interface apparatus 
according to the first embodiment of the present invention. 
10 [0066] The digital broadcast receiving system 1000 includes a recording and replaying apparatus 1100, a monitor 
1 200 for displaying images, and a remote controller 1 300 for receiving user instructions and transmitting the received 
instructions to the digital broadcast receiving apparatus 1100. 

[0067] The recording and replaying apparatus 1100 includes an infrared ray reception unit. The recording and re- 
playing apparatus 1100 detects a signal that has been transmitted from the remote controller 1 300 according to a user 

15 instruction using the infrared ray reception unit, executes such functions as receiving a digital broadcast program, 
recording a received program, and replaying a recorded program in response to the user instruction, and outputs a 
replayed Image or an image for displaying a menu for function selection to the monitor 1200. The recording and re- 
playing apparatus 1100 includes a controller 1110, a recorder 1120, a broadcast receiving antenna 1130, a receiving 
unit 1140, a coding unit 1150, a replaying unit 1160. and an output unit 1170. 

20 [0068] Here, the controller 1110 includes a CPU and a storage unit such as a memory. The controller 1110 realizes 
a execution control function for controlling the operations by each of the elements In the recording and replaying ap- 
paratus 1100 according to the signals transmitted from the remote controller 1 300. The storage unit stores a program 
for the control. More specifically, the controller 1110 receives the information on an Electric Program Guide (EPG) 
(referred to a "program table" in this specification) from the information that the receiving unit 1140 has received, and 

25 outputs the image of the program table according to the received information to the monitor 1200 via the output unit 
1170. The controller 1110 has the receiving unit 1140 receive a program on a channel that the user has designated, 
and controls the detection of programs, the presetting for tuning to programs after the detection, the recording of the 
history of watched programs, and the test of receiving condition of the broadcast receiving antenna 1130. 
[0069] Note that a program table includes the program titles, program identifiers Including broadcasting time, and 

30 other information such as, a genre, a subgenre, a program attribute, players, explanation for each of the programs. 
The program table for a week is sent once each hour. For instance, the genre of a program is sport, the subgenre Is 
succor or baseball, and the program attribute is a serial. 

[0070] In addition, the controller 1110 realizes a user interface control function, fy/lore specifically, the controller 1110 
sends the image of a menu to the output unit 1170 in order to display the menu that is a graphical user interface for 

35 the function selection by the user on the monitor 1200. 

[0071] Note that in each of the menus, a plurality of button images and a cursor are shown. The menus are classified 
into two groups, i.e., standard menus, which are included In a hierarchical structure, and auxiliary menus, each of 
which Is displayed in a specific condition. Here, a button image is an image of the characters that read the content of 
a function of the recording and replaying apparatus 1100 as the button images in the description of the prior art. 

40 [0072] The recorder 1120 includes a hard disk or a Digital Video Disc-RAM (DVD-RAM). The recorder 1120 stores 
a huge amount of Image data. 

[0073] The receiving unit 1 1 40 includes a tuner, a system stream decoder and an inr^ge and audio decoder of Motion 
Picture Expert Group (MPEG). The receiving unit 1140 receives digital broadcast programs via the broadcast receiving 
antenna 1130, demodulates the received programs, and separates a program that the user desires into packets to 
45 decode the program. In addition, the receiving unit 1140 receives a program table. 

[0074] The coding unit 1 1 50 compresses and codes images. The replaying unit 1 1 60 replays the programs recorded 
in the recorder 1120. 

[0075] On receiving data for displaying an image from the replaying unit 1160 and the controller 1110. the output unit 
1170 compounds the data as necessary and outputs the compounded data as a TV signal to the monitor 1200. 

so [0076] The remote controller 1 300 Includes a power button 1 31 0 for turn on and off the power of the recording and 
replaying apparatus 1100, a menu button 1320 for having the monitor 1200 display a standard menu for function 
selection that is a graphical user interface, cursor buttons 1330, 1340, 1350, and 1360 for moving the cursor on the 
monitor 1200, a determination button 1370 for determining the selection of a button image, and an auxiliary menu 
operation button 1 380. When the user presses one of the buttons, the remote controller 1 300 transmits a signal to the 

55 recording and replaying apparatus 1100 for identifying the pressed button. 

[0077] The auxiliary menu operatton button 1 380 is used for operating auxiliary menus. The auxiliary menu operation 
button 1380 includes auxiliary cursor buttons 1382 and 1383 for moving a cursor that is used for selecting a button 
image on an auxiliary menu and an auxiliary determination button 1 381 for determining the selection of a button image. 
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[0078] The functional structure of the controller 1110, which realizes an execute control function and a user interface 

control function, will be explained below. 

[0079] Fig. 4 is a functional block diagram of the controller 1110. 

[0080] The execute control function and the user interface control function of the controller 1110 are realized by an 
s operation receiving unit 1411, an execution instruction/apparatus condition information storage unit 1412, a function 
execution controller 1413, an image data storage unit 1414, an auxiliary menu display timing controller 1415, an aux- 
iliary menu Item determining unit 1418. an auxiliary menu item information storage unit 1417, an auxiliary menu con- 
troller 1418, an auxiliary menu control receiving unit 1419, and an auxiliary menu display unit 1420. 
[0081] The image data storage unit 1414 stores image data in advance for displaying screens for receiving setting 
JO information from the user that is necessary for executing standard menus, auxiliary menus, and the functions. 

[0082] The operation receiving unit 1411 detects a signal transmitted from the remote controller 1300 and refers to 
standard menu display control information, which will be described later, stored in the execution instruction/apparatus 
condition information storage unit 1412. The operation receiving unit 1411 Instructs the function execution controller 
141 3 to move the cursor according to user instruction. When the user selects one button image, the operation receiving 
J5 unit 1411 stores execute control information, which will be described later, in the execution instruction/apparatus con- 
dition information storage unit 1412 according to the selected button image. 

[0083] The function execution controller 1413 refers to the execution instruction/apparatus condition Information 
storage unit 1412, decodes the user instruction, and controls each of the elements in the recording and replaying 
apparatus 1 1 00 as necessary. By doing so, the function execution controller 1 41 3 executes the function corresponding 
20 to the user instruction, and updates the information stored in the execution instruction/apparatus condition information 
storage unit 1 41 2. Meanwhile, the function execution controller 1 41 3 refers to the image data stored in the image data 
storage unit 1414 as necessary, and sends the image of the screen to the output unit 1170 for receiving the setting 
information from the user that is necessary to execute standard menus and the functions. 

[0084] Here, the content stored in the execution instruction/apparatus condition information storage unit 1412 will 

25 be explained with reference to Figs. 5 and 6. 

[0085] Fig. 5 shows the data that the execution instruction/apparatus condition Information storage unit 1 41 2 stores. 
Fig. 6 shows the standard menu hierarchical structure information Included in the standard menu display control infor- 
mation that is stored In the execution instruction/apparatus condition information storage unit 1412. 
[0086] As shown in Fig. 5, the execution instruction/apparatus condition information storage unit 1412 stores the 

30 execute control information Including the standard menu display control information, apparatus condition information, 
execution instruction information, and execution management Information. 

[0087] The standard menu display control Information includes the standard menu hierarchical structure Information 
(refer to Fig. 6) and the information that is updated by the operation receiving unit 1411 and holds the item in a menu 

indicated by the cursor. 

35 [0088] The apparatus condition information includes the information that shows the available disk space, default disk 
information that shows a selected disk in the recorder 1120, and the information on the image quality in recording. The 
apparatus condition information is referred to by the function execution controller 1413. 

[0089] The execute control information is stored by the operation receiving unit 1 411 or the auxiliary menu controller 
1418. The execute control information Is referred to and updated by the function execution controller 1413. 

40 [0090] The execution Instruction information in the execute control information is the information the value of which 
Is defined according to user instruction. The executbn instruction infornaatlon Is necessary to the execution of any 
function of the recording and replaying apparatus 1100. The execution instruction information includes a large item, a 
middle item, a small item, a setting 1, a setting 2, a setting 3, an object 1, an attribute 1, an object 2, an attribute 2, 
and a sub command. Note that the sub command is a command description, in which a function that is to be immediately 

45 executed is described. The sub command is needed only when one piece of the information of the setting 1 to the 
attribute 2 is determined by the execution of the sub command. 

[0091] Here, the functions to be executed are classified into large, middle, and small items according to the hierar- 
chical structure of the standard menus, and values are defined in advance for identifying the functions. In each of the 
large, middle, small items in the execution instruction information, one of the defined values is stored according to user 
50 selection. 

[0092] The settings 1 , 2, and 3 correspond to the large, middle, and small items, respectively The objects 1 and 2 
are the information that show the objects of the function to be executed. The attributes 1 and 2 are the information that 
show the characteristics and the like of the objects 1 and 2, respectively 

[0093] The execution management information in the execute control information corresponds to the execution in- 
55 struction information that has been described, and shows the present operating stage of the function that is shown by 
the execution instruction information. More specifically, the execution management information has a value showing 
each of "not decoded", "wait for execution", "now executing", and "executed". Here, the "not decoded" means that the 
execution Instruction information has not been decoded. 
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[0094] Note that a plurality of pieces of execute control Information may be stored In the execution instruction/appa- 
ratus condition information storage unit 1 41 2. When the number of pieces of execute control information for which the 
execution management information has the value showing "executed" is larger than a predetermined number, the 
function execution controller 1413 deletes a piece of execute control information for which the function was executed 
s earlier than any other piece of execute control information. 

[0095] Fig. 7 shows an example of data that the execution instruction/apparatus condition information storage unit 
1412 stores. 

[0096] Fig. 7 shows a specific example of the information stored in the execution instruction/apparatus condition 
information storage unit 1412 except for the standard menu display control Information. More specifically, Fig. 7 shows 
10 the condition that one piece of execute control information is stored in the execution instruction/apparatus condition 
information storage unit 1412. 

[0097] In this example, the large item in the execution instruction infornfiatlon is "preset for recording", the middle 
item is "detect and preset", the small Item Is "detect from program table", the setting 1 is "high quality image", the 
settings 2 and 3 are "nothing", the object 1 Is "XY2Z... (for instance, the date for recording)", the object 2 Is "disk 1 ". 

'5 the attribute 2 is "nothing", and no sub command Is set. The execution management information shows "wait for exe- 
cution". In the apparatus condition information, the available disk space is "10GByte", the default disk information is 
"disk 1 ", and the image quality in recording Is "high quality". Note that actually represented by binary codes, the "preset 
for recording" and the like are represented by characters in the interest of simplicity of explanation. 
[0098] The elements that realize the functions of the controller 1110 will be again explained below. 

20 [0099] The auxiliary menu display timing controller 1 41 5 is informed of the wait for and the completion of the execution 
of the function corresponding to user instruction by the function execution controller 1413. The auxiliary menu display 
timing controller 1415 instructs the auxiliary menu item determining unit 1416 to start determining the Items of an 
auxiliary menu with the timing related to the information from the function execution controller 1413. 
[0100] The auxiliary menu item determining unit 1416 determines the items of the auxiliary menu, and Instructs the 

2S function execution controller 1 41 3 to update the apparatus condition information in the execution instruction/apparatus 
condition information storage unit 1412. The auxiliary menu item determining unit 1416 determines the items on the 
auxiliary menu by referring to the execution Instruction/apparatus condition information storage unit 1 41 2 and the aux- 
iliary menu item information storage unit 1417, and informs the auxiliary menu controller 1418 of the determined aux- 
iliary menu items. 

30 [0101] Note that the auxiliary menu item determining unit 1 416 determines as many auxiliary menu items as possible 
subject to a maximum of five. As a result, the auxiliary menu has five auxiliary menu Items at the maximum. 
[0102] On Instructed by the auxiliary menu controller 1418, the auxiliary menu display unit 1420 obtains the Image 
that is necessary to display the auxiliary menu on the monitor 1200 by referring to the Image data storage unit 1414, 
and sends the obtained Image to the output unit 1170. 

35 [0103] The auxiliary menu control receiving unit 1419 detects user operation of the auxiliary menu operation button 
1380 and informs the auxiliary menu controller 1418 of the detected operation. 

[0104] The auxiliary menu controller 1 41 8 instructs the auxiliary menu display unit 1420 to display the auxiliary menu 
including the items that have been informed of by the auxiliary menu item determining unit 1416. The auxiliary menu 
controller 1 41 8 instructs the auxiliary menu display unit 1 420 to move the cursor according to the user instruction that 
^0 has been informed of by the auxiliary menu control receiving unit 1419. When the user instructs to select a button 
Image on the auxiliary menu, the auxiliary menu controller 1 41 8 stores the execute control information in the execution 
Instruction/apparatus condition information storage unit 1412 for realizing the function of the recording and replaying 
apparatus 1 1 00 corresponding to the selected button image. 

[0105] The data that the auxiliary menu item information storage unit 1417 stores will be explained below. 
^ [0106] Fig. 8 shows the data that the auxiliary menu Item information storage unit 1417 stores. 

[0107] As shown in Fig. 8, the auxiliary menu item information storage unit 1417 stores in advance auxiliary menu 
item information, each piece of which is necessary to control the display of an auxiliary menu item, for a plurality of 

predetermined prospective auxiliary menu items. 

[0108] The auxiliary menu Item information is referred to by the auxiliary menu item determining unit 1416 and the 
so auxiliary menu controller 1418. A piece of auxiliary menu item infonnation includes an auxiliary menu item name, a 
piece of auxiliary menu execution instruction Information, an auxiliary menu Item priority, and an auxiliary menu item 

display condition. 

[0109] An auxiliary menu item name is character string that is written on a button image on the auxiliary menu, and 
shows the content of the function of the function of the recording and replaying apparatus that Is executed when the 
55 user select the button image. 

[0110] A piece of auxiliary menu execution instruction information instructs the recording and replaying apparatus 
1100 of the execution related to the function that is to be executed when the corresponding button Image is selected 
by the user. A piece of auxiliary menu execution instruction information has the same content as a piece of execution 
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instruction infornnation that is stored in the execution instruction/apparatus condition information storage unit 1412. 
[0111] An auxiliary menu Item display condition shows the Informatbn on the condition on which the corresponding 
auxiliary menu item is displayed. An auxiliary menu Item display condition shows the condition on the contents of a 
piece of execute control information and apparatus condition information. 
5 [0112] An auxiliary menu item priority is referred to when the auxiliary menu item determining unit 1 416 determines 
whether the corresponding prospective auxiliary menu item is selected as an auxiliary menu item. Prospective auxiliary 
menu items are selected In order of the auxiliary menu item priorities. 

[0113] Figs. 9 and 10 show examples of the auxiliary menu item infonrnation stored in the auxiliary menu Item Infor- 
mation storage unit 1417. 

10 [0114] In an auxiliary menu item information 1 81 0 in Fig. 9, the auxiliary menu item name is a character string 'deletion 
of replayed file". In the auxiliary menu execution instruction information, the large item is "setting", the middle item is 
"file deletion", the small item is 'deletion of replayed file", and the settings 1. 2, and 3. the attributes 1 and 2, and the 
objects 1 and 2 are "nothing*. The auxiliary menu item priority is "1 and the auxiliary menu Item display condition is 
"large item==replay, execution management information==executed, available disk space<2GByte". 

15 [0115] Here, the "==" shows equivalence. While actually represented by binary data, the data except for the auxiliary 
menu item name are represented by characters in the interest of simplicity of explanation. 

[0116] The auxiliary menu item information 1810 shows that the replay has been executed and the available disk 
space Is less than 2 gigabytes, i.e.. the auxiliary menu item information 1810 means that an item "deletion of replayed 
file" may be shown on the auxiliary menu. 

20 [0117] In an auxiliary menu item information 1820 shown in Fig. 10, the auxiliary menu item name is a character 
string "preset for recording of next episode". In the auxiliary menu execution instruction information, the large item is 
"preset for recording", the middle item is "detect and preset", the small item is "detect from program table", the setting 
" is "getmode", the settings 2 and 3 are "nothing", the object 1 is "search", the attribute 1 is "nothing", the object 2 is 
"getdlsk", the attribute 2 is "nothing", and the sub command is "getmode. search (?objectl, EPG, NEXT), getdisk (de- 

25 fault)". The auxiliary menu item priority is "2", and the auxiliary menu item display condition is "(large item==preset for 
recording, execution management information==wait for execution or executed) or (large item==replay, execution man- 
agement information==executed)". 

[0118] Here, in the sub command, the function that is to be immediately executed in order to supplement the infor- 
mation in the large item to the attribute 2. More specifically, the "getmode" is a function of obtaining a set image quality 

30 in recording. The "search(?object 1 , EPG, NEXT)" is a function of retrieving the information on the next episode of the 
program that is shown by the "?object 1" from the program table of digital broadcast programs. The "getdlsk(default) 
" is a function of obtaining the default disk information. Note that the ■?object 1 " is the copy of the value of the object 
1 in a piece of execution instruction information when the auxiliary menu item determining unit 1416 judges that the 
content of the piece of execution instruction information in the execution instruction/apparatus condition information 

35 storage unit 1412 satisfies the auxiliary menu item display condition. 

[0119] The auxiliary menu Item information 1820 means that the item "preset for recording of next episode" may be 
displayed on the auxiliary menu when the execution of a preset for recording is waited for or completed, or when a 
replay is executed. 

40 (Operation) 

[0120] The operation by the digital broadcast receiving system 1000, the structure of which has been described, will 

be explained below. 

[0121] The recording and replaying apparatus 1100 records and replays digital broadcast programs under the control 
45 of the controller 1110. 

[01 22] The recording by the recording and replaying apparatus 1 1 00 is realized when the receiving unit 1 1 40 receives 
the radio waves that multiplex the information on broadcast programs and the like via the broadcast receiving antenna 
1 1 30, extracts the information on a desired program, compresses and encodes the extracted information in the coding 
unit 1150, and stores the information in the recorder 1120. 
50 [0123] The replaying by the recording and replaying apparatus 1100 Is realized when the replaying unit 11 60 fetches 
the information on a program that is stored in the recorder 1120, and outputs the fetched information to the monitor 
1200 via the output unit 1170. 

[0124] The controller 1110 displays a standard menu and a following screen for user input to receive user instruction, 
and execute a function such as recording and replaying according to the user instruction. In addition, the controller 
55 1110 predicts the function that is to be desired by the user next, and provides a user interface as the auxiliary menu 
so that the user would select the function. Here, "to predict" means to identify a function that is not directly indicated 
by the user 

[0125] The operation by the digital broadcast receiving system 1000 will be described in detail, focusing on the 
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operation by the controller 1110 that displays the auxiliary menu and has the recording and replaying apparatus 1100 
execute functions of recording, replaying, and the like. 

[0126] In the explanation, the user Is supposed to preset the recording and replaying apparatus 1100 to record a 

serial. 

5 [0127] When the user presses the menu button 1320 on the remote controller 1300. the remote controller 1300 
transmits the signal for indicating that the menu button 1320 has been pressed. The operation receiving unit 1411 in 
the controller 1110 detects the signal, and demands the function execution controller 1 41 3 to display a standard menu. 
[0128] Then, the function execution controller 1413 transmits the Image for displaying the standard menu to the 
output unit 1170 using image data that is stored In the image data storage unit 1414. 

10 [0129] As a result, the output unit 1 1 70 outputs a TV signal to the monitor 1 200, and the standard menu is displayed 
on the monitor 1200. 

[0130] When the user presses the cursor button 1350 after the display of the standard menu on the monitor 1200, 
the remote controller 1300 transmits the signal for Indicating that the cursor button 1350 has been pressed. The op- 
eration receiving unit 1411 in the controller 1110 detects the transmitted signal, updates the information that holds the 
IS cursor location in the standard menu display control information that is stored in the execution instruction/apparatus 
condition information storage unit 1412, and instructs the function execution controller 1413 to move the cursor 
[0131] On receiving the instruction, the function execution controller 1413 displays the Image on the monitor 1200 
via the output unit 1170 in which the cursor moves down one line. 

[0132] As a result, the monitor 1200 displays the standard menu in a screen condition 2010 in Fig. 11. 

20 [0133] Fig. 1 1 shows the change of the screen on the monitor 1200 according to user instruction. 

[0134] When the user presses the determination button 1370 after the monitor 1200 has displayed the standard 
menu in the screen condition 201 0, the remote controller 1 300 transmits the signal for Indicating that the determination 
button 1370 has been pressed. The operation receiving unit 1411 in the controller 1110 detects the transmitted signal, 
refers to the standard menu display control information that is stored in the execution instruction/apparatus condition 

25 information storage unit 1412. and demands the function execution controller 1413 to display the function items that 
are included in the menu just below the item "preset for recording" (refer to Fig. 6), which the cursor indicates. 
[0135] On receiving the demand, the function execution controller 141 3 displays a standard menu In a screen con- 
dition 2020 in Fig. 11 on the monitor 1200 via the output unit 1170 using Image data stored In the image data storage 
unit 1414. 

30 [0136] The processing below is similar to the processings that have been described. When the user presses the 
determination button 1370, the function Items that are included In the menu just below the item "detect and preset" 
(refer to Fig. 6) are displayed on the monitor 1200. When the user further presses the cursor button 1350. the screen 

on the monitor 1200 is in a screen condition 2030 in Fig. 11. 

[0137] When the monitor displays the screen in the screen condition 2030, and when the user presses the determi- 
35 nation button 1370, the operation receiving unit 1411 detects the signal transmitted from the remote controller 1300, 
refers to the standard menu display control Information that is stored In the execution Instruction/apparatus condition 
information storage unit 1412, and stores a piece of execute control information in the execution instruction/apparatus 

condition information storage unit 1412. 

[0138] Here, the piece of execute control information that the operation receiving unit 1411 stores in the execution 
40 instruction/apparatus condition information storage unit 1412 (refer to Fig. 5) includes execution management infor- 
mation "not decoded" and execution instruction infornnation in which the large item is "preset for recording", the middle 
item is "detect and preset", the small item is "detect from program table", and the settings 1, 2, and 3. the objects 1 

and 2, and the attributes 1 and 2 are "nothing". 

[0139] The piece of execute control information that the operation receiving unit 1411 has stored in the execution 
^ instnjctlon/apparatus condition information storage unit 1412 is decoded by the function execution controller 1413. 
[0140] The operation by the function execution controller 141 3 will be explained below. 

[0141] Fig. 12 is a flowchart showing the operations by the function execution controller 1413 related to execute 

control information. 

[01 42] The function execution controller 1413 performs other operations that are not shown in Fig, 1 2 in parallel with 

50 the operations shown In Fig. 12. 

[0143] When at least one piece of execute control information is stored in the execution instruction/apparatus con- 
dition information storage unit 1412, the function execution controller 1413 focuses on one piece of execute control 
information (step SI 01) and judges whether the execution management information in the piece of execute control 
information is "not decoded" (step S102). 

55 [01 44] When the execution management information is "not decoded", the function execution controller 1413 judges 
whether a sub command is included in the execution instruction information in the execute control information (step 
S107). When a sub command is included in the execution instruction information, the function execution controller 
1 41 3 performs a function according to the sub command and updates the execution instruction information (step SI 08). 
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[0145] Atter the processes related to the sub command (steps S107 and S108). the function execution controller 
141 3 judges whether the execution instruction information is imperfect and whether user instruction is necessary to 
complete the execution instruction information (step S103). For the judgement at step S103, the function execution 
controller 1413 uses the fact that the value corresponding to "nothing" is set for the setting 1 and the contents of the 

5 large, middle, and small items In the execution Instruction information. 

[0146] When judging that user instruction is necessary at step SI 03, the function execution controller 141 3 obtains 
the Image data from the image data storage unit 1414 and the receiving unit 1140 that is necessary to the function 
identified by the execution instruction information in order to encourage user operation, and displays the image for 
Input on the monitor 1200 via the output unit 1170. The function execution controller 1413 obtains the information that 

10 the user has Input via the operation receiving unit 1411 (step SI 04). 

[0147] At step SI 04. on receiving user Input, the operation receiving unit 1411 updates the contents of the setting 1 
to the attribute 2 In the execution instruction information in the execute control information stored in the execution 
instruction/apparatus condition information storage unit 1412 according to the user input. When the user selects the 
cancellation of the function, the operation receiving unit 1411 informs the function executbn controller 1413 of the 

15 cancellation. 

[0148] When informed of the function cancellation by the user from the operation receiving unit 1411 at step S104 
(step SI 05), the function execution controller 1413 deletes the execute control information that has been focused on 
(step SI 06), and focuses on another piece of execute control information (step 8101). 

[0149] When judging that user Instruction is not necessary at step S103. or when the cancellation is not selected at 
20 stepSIOS, the function execution controller 1413 judges whether the function that Is shown by the large, middle, and 
small items in the execution instruction information is a preset function, which is not immediately executed (step SI 09). 
For Instance, a "preset for recording" Is a function that Includes the waiting state before the recording. As a result, the 
result of the judgement at step SI 09 for a "preset for recording" Is "yes". 

[0150] When judging that the function is a preset function at step SI 09. the function execution controller 1413 chang- 
25 es the execution management Information as "wait for execution", and informs the auxiliary menu display timing con- 
troller 1415 of the timing In starting the process for displaying the auxiliary menu (step S111). The process returns to 
step SI 01. 

[0151] Note that the timing of the change of the execution management information to "wait for execution" corre- 
sponds to the timing of the completion of the user operation of the recording and replaying apparatus 1100 for one 
30 function. 

[0152] Meanwhile, when judging that the execution management information is not "not decoded" at step 8102, the 
function execution controller 141 3 judges whether the execution management information is "wait for execution" and 
whether it is the time for execution from the content of the execution Instruction information (step 8112). 
[0153] When the result of the judgement at step S112 is "no", the process returns to step S101 . 

35 [0154] When the result of the judgement at step S112 is "yes", the function execution controller 141 3 changes the 
execution management information in the execute control Information to "now executing" (step 811 3), and executes 
the function that is identified by the execution instruction information in the execute control information (step 8114). 
[0155] After executing the function at step S1 14, the function execution controller 1413 changes the execution man- 
agement information in the execute control information to "executed" (step S115), and Informs the auxiliary menu 

40 display timing controller 1415 of the timing in starting the process for displaying the auxiliary menu (step S116). The 
process returns to step 8101 . 

[0156] Note that the function execution controller 1 41 3 detects the completion of the function by the elements in the 
recording and replaying apparatus 1100 by receiving signals from the elements that inform of the completion of the 
function, for Instance. After detecting the completion of the function, the function execution controller 1413 sets the 

45 execution management information as "executed". 

[0157] The operation by the function execution controller 1 41 3 under the condition described betow will be explained. 
The condition is that the user selects the function "detect from program table" in the screen in the screen condition 
2030 in Fig. 11, and the operation receiving unit 1411 stores the execute control Information that has been described 
in the execution Instruction/apparatus condition information storage unit 1412. 

50 [0158] The operation by the function execution controller 1413 focuses on the execute control information in which 
the large item in the execution instruction information is "preset for recording", the middle item is "detect and preset", 
the small item is "detect from program table", and the setting 1 to the attribute 2 are "nothing", and the execution 
arrangement information is "not decoded" (step SI 01). 

[0159] The function execution controller 1413 judges that the execution management information is "not decoded" 
55 (step SI 02) and that no sub command Is included (step SI 07). Finding that the setting 1 and the like are "nothing", 
the function execution controller 141 3 judges that user instruction is necessary (step 81 03). Responding to the small 
item "detect from program table", the function execution controller 1413 controls the receiving unit 1140 so that the 
program table data would be obtained and an Image would be generated. The function execution controller 1413 
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displays the image on the monitor 1 200 via the output unit 11 70, and receives user operation via the operation receiving 
unit 1411 (stepS104). 

[0160] As a result, the screen In an Image condition 2040 In Fig. 13 Is displayed on the monitor 1200. 
[0161] Fig. 1 3 shows the change of the screen on the monitor 1 200 after the screen change shown in Fig. 11 . 

5 [0162] The screen condition 2040 in Fig. 1 3 includes date change button images 2041 and 2042 for changing date, 
and time slot change button images 2043 and 2044 for scrolling the program table up and down on the screen. 
[0163] The user moves the cursor location on the screen at the date change button image 2041 or 2042 using the 
cursor buttons 1330, 1340, 1350, and 1360 on the remote controller 1300. and sets a desired date by pressing the 
determination button 1 370. In a similar way, the time slot change button image 2043 or 2044 is selected and desired 

10 time slots are set. As a result, programs displayed on the screen are changed. The program display change is realized 
when the operation receiving unit 1411 directly instructs the function execution controller 1413 to change images. 
[0164] Suppose that the user wants to preset the recording and replaying apparatus 11 00 to record a serial "XYZZ". 
positions the cursor location at the arrow in the screen condition 2040 in Fig. 1 3 using the cursor buttons 1 330, 1 340, 
1350, and 1360 on the remote controller 1300, and presses the determination button 1370. 

15 [0165] On receiving the user operation, the operation receiving unit 1411 updates the execution instruction informa- 
tion stored in the execution instruction/apparatus condition information storage unit 1412 according to the Information 
that the user has selected, and has the monitor 1 200 display the innage in an screen condition 2050 in Fig. 1 3 via the 
function execution controller 1413. 

[0166] When the user selects the "confirmation" in the image in the screen condition 2050, the operation receiving 
20 unit 1411 receives the user selection and informs the function execution controller 1413 of the selection. 

[0167] The function execution controller 1413 judges that the "cancellation" has not selected (step SI 05), refers to 
the execute control information, and judges that a preset function has been instructed (step SI 09). 
[0168] In this example, the user selects the functions "preset for recording", "detect and preset", and "detect from 
program table". As a result, the function execution controller 1413 judges that a preset function has been selected at 
25 step SI 09, changes the execution management information In the execute control information stored in the execution 
instruction/apparatus condition information storage unit 1 41 2 to "wait for execution" (step S11 0). and gives the auxiliary 
menu display controller 1 41 5 the timing in starting the process for displaying the auxiliary menu (step S11 1 ). Then, the 
process returns to step SI 01. 

[0169] As a result, the execute control information is that shown in Fig. 7. 
30 [0170] The auxiliary menu display timing controller 1415, which has been given the timing in starting the process for 
displaying the auxiliary menu by the function execution controller 1413, instructs the auxiliary menu item determining 
unit 1416 to determine the auxiliary menu items that are to be Included on the auxiliary menu when a predetermined 

time (for Instance, a few seconds) has elapsed since the timing. 

[0171] The operation related to the display of the auxiliary menu by the controller 1110 after the auxiliary menu item 
35 determining unit 1 41 6 is Instructed to determined the auxiliary menu items by the auxiliary menu display timing controller 
1415 will be explained below. 

[0172] Fig. 1 4 is a flowchart showing the process related to the display of the auxiliary menu by the controller 111 0. 

[0173] When receiving the instruction to determine the auxiliary menu items on the auxiliary menu from the auxiliary 

menu display timing controller 1415, the auxiliary menu item determining unit 1416 demands the function execution 
40 controller 1413 to detect the present apparatus condition and to update the apparatus condition information in the 

execution Instruction/apparatus condition information storage unit 1412 (refer to Fig. 5) (step S201). 

[0174] On receiving the demand, the function execution controller 1 41 3 controls the elements In the recording and 

replaying apparatus 1100 to detect the present condition of the recording and replaying apparatus 1100, updates the 

apparatus condition Information according to the detected apparatus condition. 
45 [0175] After the update of the apparatus condition information, the auxiliary menu item determining unit 1416 refers 

to the execute control Information and the apparatus condition information stored in the execution instruction/apparatus 

condition Information storage unit 1412, and detects the auxiliary menu item information whose auxiliary menu item 

display conditions are satisfied in the auxiliary menu item information stored in the auxiliary menu item Information 

storage unit 1417 (refer to Fig. 8) (step S202). 
50 [0176] After the detection, the auxiliary menu Item determining unit 1416 refers to the auxiliary menu item priorities 

of the detected auxiliary menu item Information, and select as many auxiliary menu items as possible within five to 

determine the auxiliary menu Items displayed on the auxiliary menu (step S203). 

[01 77] After determining the auxiliary menu items, the auxiliary menu item determining unit 1 41 6 informs the auxiliary 
menu controller 1418 of the information for identifying the selected auxiliary menu items. Note that when finding that 
55 the Information indicating to fetch a piece of Information in the execution Instruction Information that satisfies the con- 
dition during the detection of the desired auxiliary menu item information, for instance. "?object 1" (refer to Fig. 10). 
the auxiliary menu Item determining unit 1416 Informs the auxiliary menu controller 1418 of the value of the piece of 
Information. 
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[0178] Meanwhile, when finding no desired auxiliary menu item information at step S202, the auxiliary menu item 
determining unit 1416 informs the auxiliary menu controller 1418 of nothing. Then, the process related to the display 
of the auxiliary menu is completed. 

[0179] On receiving information from the auxiliary menu item determining unit 1416, the auxiliary menu controller 
s 1418 refers to the auxiliary menu item information storage unit 1 41 7, and displays the image of the auxiliary menu via 
the auxiliary menu display unit 1420 (step S204). 

[0180] More specifically, image data is transmitted from the auxiliary menu display unit 1 420 to the output unit 11 70. 
and a TV signal is output from the output unit 1170 to the monitor 1200. As a result, the auxiliary menu Is displayed on 

the monitor 1200. 

10 [0181] After displaying the auxiliary menu, the auxiliary menu controller 1 41 8 receives user operation via the auxiliary 
menu control receiving unit 1419, and controls cursor movement and the like according to the user operation. The 
auxiliary menu controller 1 41 8 judges whether the user has selected a button image on the auxiliary menu (step S205). 
When receiving no user operation after 10 seconds have lapsed since the display of the auxiliary menu or a cursor 
movement (step S206), the auxiliary menu controller 1418 deletes the auxiliary menu (step S207). 

'5 [0182] Meanwhile, when judging that the user has selected one button image (step S205), the auxiliary menu con- 
troller 1 41 8 stores execute control information in the execution instruction/apparatus condition information storage unit 
1 41 2 so that the function that has been selected by the user would be executed. The function execution controller 1 41 3 
controls the elements in the recording and replaying apparatus 1 1 00 according to the stored execute control information 
to execute the selected function (step S208). 

20 [0183] Note that when storing the execute control information in the execution instruction/apparatus condition infor- 
mation storage unit 1 41 2 at step S208. the auxiliary menu controller 1 41 8 uses the auxiliary menu execution Instruction 
information in the auxiliary menu item Information as the execution nnanagement information and "uncoded" as the 
execution management Information. 

[0184] The process shown in the flowchart in Fig. 14 will be explained with reference to a specific example. 

2$ [0185] Suppose that the auxiliary menu item information storage unit 1417 stores a plurality of pieces of auxiliary 
menu item information as well as the auxiliary menu item information shown In Figs. 9 and 10. 
[0186] On receiving an instruction from the auxiliary menu display timing controller 1415, the auxiliary menu Item 
determining unit 1416 demands the function execution controller 141 3 to detect the present apparatus condition (step 
S201). At this time, the inside condition of the execution instruction/apparatus condition information storage unit 1412 

30 Is as shown in Fig. 7. 

[0187] Fig. 7 shows that the recording and replaying apparatus 1100 is preset to record the serial "XYZZ", and the 

recording Is waited for. 

[01 88] At step S202, the auxiliary menu item determining unit 1 41 6 detects the auxiliary menu item information whose 
auxiliary menu item display conditions are satisfied. As a result, the auxiliary menu item information shown in Fig. 9, 

3S whose auxiliary menu Item display condition is not satisfied, is not detected. On the other hand, the auxiliary menu 
item information shown in Fig. 10, whose auxiliary menu item display condition is satisfied, is detected. 
[0189] After the detection, the auxiliary menu item determining unit 1416 refers to the auxiliary menu item priorities 
in the detected auxiliary menu item information, selects up to five pieces of auxiliary menu item information in order of 
priority, and notifies the auxiliary menu controller 1418 of the selected auxiliary menu item information. 

40 [0190] Note that the information "?object 1" is included in the auxiliary menu item information shown in Fig. 10. As 
a result, the auxiliary menu controller 1418 is notified of the object 1 "XYZZ... (for instance, the date for recording)' in 
the execution Instruction infomiation shown in Fig. 7 as the corresponding information by the auxiliary menu item 
determining unit 1416. 

[0191] On receiving the notification from the auxiliary menu item determining unit 1416, the auxiliary menu controller 
1418 displays the image of the auxiliary menu via the auxiliary menu display unit 1420 (step S204). 
[0192] As a result, the screen In a screen condition 2060 in Fig. 16 is displayed on the monitor 1200. Each of the 
button Images on the auxiliary menu corresponds to each piece of auxiliary menu item information that the auxiliary 
menu item determining unit 141 6 has determined. The button Images are displayed on the screen in order of decreasing 
priority of the corresponding auxiliary menu item information. 
so [0193] Fig. 1 5 shows the change of the screen displayed on the monitor 1 200. 

[0194] When the screen in the screen condition 2060 is displayed on the monitor 1 200, the user operates the auxiliary 
menu operation button 1 380 to move the cursor on the screen and selects a button image. 

[0195] When the user selects a button image "preset for recording of next episode", the auxiliary menu controller 
1418 receives the user selection via the auxiliary menu control receiving unit 1419 (step S205). The auxiliary menu 
ss controller 1418 stores execute control information in the execution instruction/apparatus condition information storage 
unit 1412 using the auxiliary menu Item information corresponding to the selected button image (refer to Fig. 10) to 
have the function execution controller 1413 execute a desired function (step S208). 

[01 98] Here, the execute control information that is stored in the execution instruction/apparatus condition Information 
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storage unit 1412 has execution management information "not decoded" and execution instruction information, which 
is the auxiliary menu execution Instruction information in Fig. 10 In which the "?object 1" Is replaced by "XYZZ... (for 
instance, the date for recording)". 

[0197] On receiving the execute control information, the function execution controller 1413 performs the process 
5 shown in the flowchart in Fig. 12. 

[0198] More specifically, the function execution controller 1413 focuses on the execute control information (step 

S101), judges that the execution management information is "not decoded" (step S102), judges that a sub command 

is included in the execution instruction information (step SI 07). and executes the sub commands (step SI 08). 

[0199] As a result, the setting 1 In the execution instruction information in the execute control information is changed 
10 to "high quality image" due to the execution of the "getmode", the object 2 is changed to "disk 1 " due to the execution 

of the ''getdisk(default)", and object 1 Is changed to the infomnation on the next episode of the serial XYZZ due to the 

execution of "search('XYZZ... (for Instance, the date for recording)", EPG, NEXT)". 

[0200] After the sub command execution, the function execution controller 141 3 judges whether user Instruction Is 
necessary (step 8103). In this case, it is judged that user confirmation is necessary since the large item is "reset for 
IS recording" although it is judged that no further information from the user is necessary since a value is given to the 
setting 1 , so that a screen for confirmation is displayed (step 81 04). 

[0201] As a result, the screen In the screen condition 2070 in Fig, 15 Is displayed on the monitor 1200. The process 
after step 8104 is the same as the process after the screen in the screen condition 2050 in Fig. 1 3 Is displayed on the 
monitor 1200, which has been described. The process after step 8104 will not be explained in the interest of simplicity. 

20 [0202] Meanwhile, when the recording for which the recording and replaying apparatus 1 1 00 has been preset starts 
to be performed, the function execution controller 1413 judges affirmatively at step S1 12 in the flowchart in Fig. 12. 
The function execution controller 1413 changes the execution management information In the execute control infor- 
mation to "now executing" (step 811 3), execute the recording (step 811 4), changes the execution management Infor- 
mation into "executed" (step 8115). and gives the auxiliary menu display timing controller 1415 the timing in starting 

2S the process for displaying the auxiliary menu (step S1 16). 

[0203] Note that when the available disk space of the recorder 1120 is less than "2GByte" after the user replays the 
recorded serial due to standard menu operation and the like, the auxiliary menu item information satisfies the condition, 
so that the button image "deletion of replayed file" is displayed on the auxiliary menu. 

[0204] As a result, the button image "deletion of replayed file", which is normally displayed on the screen after several 
30 operations by the user for standard menus (refer to Fig. 6), is automatically displayed after the replay of a recorded 
serial. 

[0205] As has been described, the digital broadcast receiving apparatus 1 1 00 has the auxiliaiy menu Item detemriining 
unit 1416 detect the functions that the user is supposed to select next according to the present apparatus condition 
and the operation that has been performed by the user, and displays the detected functions with a predetermined timing 
3S that the user is supposed to perform the next operation. As a result, the user may Instruct the execution of a function 
only by selecting the function from the menu. 

[0206] In other words, the digital broadcast receiving apparatus 1100 predicts functions that will be desired by the 
user, and provides a graphical user Interface for having the user instruct the execution of a function without user request. 

40 (Second Embodiment) 

[0207] A user interface apparatus according to the second embodiment of the present invention will be described 
with reference to Figs. 16 to 24. 

45 (Structure) 

[0208] Fig. 1 6 shows a structure of a digital broadcast receiving system 3000 equipped with a user interface apparatus 
according to the second embodiment of the present invention. 

[0209] The digital broadcast receiving system 3000 and the digital broadcast receiving system 1 000 are almost the 
so same. The digital broadcast receiving system 3000 differs from the digital broadcast receiving system 1000 only In 
including a controller 3110 instead of the controller 1110. The controller 3110 has a different user Interface control 

function from the controller 1110. 

[0210] The elements that are given the same numerical numbers in the digital broadcast receiving system 3000 and 
in the digital broadcast receiving system 1000 (refer to Fig. 3) have the same functions. The explanation below will 
55 focus on the elements having different functions in the Interest of simplicity. 

[0211] The digital broadcast receiving system 3000 Includes a recording and replaying apparatus 3100, which Is the 
same as the recording and replaying apparatus 1100 apart from including the controller 3110 instead of the controller 
1 1 1 0, a monitor 1 200 for displaying images, and a remote controller 1 300 for receiving user operation and transmitting 
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the user operation to the recording and replaying apparatus 1100. 

[0212] The recording and replaying apparatus 3100 detects a signal transmitted from the remote controller 1300 
according to user operation, executes a function such as receiving and recording a digital broadcast program and 
replaying a recorded program in response to the user operation, and outputs an image tor displaying a replayed program 
5 and a menu for function selection on the monitor 1200. The recording and replaying apparatus 3100 includes the 
controller 3110, a recorder 1120, a broadcast receiving antenna 1130, a receiving unit 1140, a coding unit 1150, a 
replaying unit 1160, and a output unit 1170. 

[0213] Here, the controller 3110 Includes a CPU and a storage unit such as a memory. The controller 3110 realizes 
a execution control function for controlling the operations by each of the elements in the recording and replaying ap- 
10 paratus 1 1 00 according to the signals transmitted from the remote controller 1 300. The storage unit stores a program 
for the control. 

[021 4] In addition, the controller 31 1 0 realizes a user interface control function. More specifically, the controller 31 1 0 

sends the image of a menu to the output unit 1170 in order to display the menu that is a graphycal user interface for 

function selection by the user on the monitor 1200. 
IS [0215] Note that in each of the menus, a plurality of button images and a cursor are shown. The menus are classified 

into two groups, i.e.. standard menus, which are included in a hierarchical structure, and auxiliary menus, each of 

which is displayed when the user presses the auxiliary determination button 1381 on the remote controller 1 300. 

[021 6] Note that the auxiliary menu In the second embodiment differs from the auxiliary menu in the first embodiment. 

The auxiliary menu in the second embodiment displays a plurality of keywords related to functions of the recording 
20 and replaying apparatus 3100 as button images. When the user select one bottn image, the auxiliary menu further 

displays button images with each of which a f unctbn of the recording and replaying apparatus 31 00 is directly selected. 

On the auxiliary menu in the present embodiment, a button innage for changing the display of keywords is Included. 

[021 7] As in the first embodiment, the auxiliary menu operation button 1 380 is used only for operating the cursor on 

the auxiliary menu. 

25 [0218] The functional structure of the controller 3110, which realizes an execute control function and a user interface 
control function, will be explained below. 

[0219] Fig. 1 7 Is a functional block diagram of the controller 31 1 0. 

[0220] The execute control function and the user interface control function of the controller 3110 are realized by an 
operation receiving unit 1411, an execution instruction/apparatus condition Information storage unit 1412, a function 

30 execution controller 1 41 3, an image data storage unit 341 4, an auxiliary menu display control information storage unit 
3417, an auxiliary menu controller 3418 that has the function corresponding to that of the auxiliary menu controller 
141 8 In the first embodiment, an auxiliary menu operation receiving unit 341 9, and an auxiliary menu display unit 3420. 
[0221] The elements that are given the same numerical numbers in the controller 3110 and in the controller 1110 
(refer to Fig. 4) have the same functions. The explanation below will focus on the elements having different functions 

35 in the interest of simplicity 

[0222] The image data storage unit 3414 stores image data in advance for displaying screens for receiving setting 
Information from the user that is necessary for executing standard menus, auxiliary menus, and the functions. 
[0223] The auxiliary menu display unit 3420 receives an Instruction from the auxiliary menu controller 3418, refers 
to the image data storage unit 341 4 to obtain the images that are necessary to display the auxiliary menu on the monitor 

<o 1 200, and sends the obtained images to the output unit 1 1 70. 

[0224] The auxiliary menu operation receiving unit 341 9 detects user operation of the auxiliary menu operation button 
1380, and sends the detected operation to the auxiliary menu controller 3418. 

[0225] The auxiliary menu controller 3418 determines the keywords and function Item names of the recording and 
replaying apparatus 3100 that are to be displayed, and instructs the auxiliary menu display unit 3420 to display the 
^ auxiliary menu that includes button images on which keywords are written and, as necessary, functions of the recording 
and replaying apparatus 3100 are written. 

[0226] The auxiliary menu controller 341 8 instructs the auxiliary menu display unit 3420 to move the cursor according 
to the user operation that has been informed of by the auxiliary menu operation receiving unit 3419. When the user 
operation is the choice of a button image on the auxiliary menu, the auxiliary menu controller 3418 instructs to change 
50 the display content of the auxiliary menu or to execute the function that the user has designated according to the user 
operation. 

[0227] More specifically the auxiliary menu controller 3418 updates the display of function items when a keyword 
has been selected by the user, and updates the keyword display when the change of keywords has been selected. 
When a function Item has been selected, The auxiliary menu controller 3418 stores execute control information in the 
55 execution instruction/apparatus condition information storage unit 1412 so that the function corresponding to the se- 
lected function Item would be executed. 

[0228] Fig. 18 shows the change of the auxiliary menu displayed on a part of the monitor 1200. 

[0229] An auxiliary menu condition 401 0 shows the condition of the auxiliary menu that is displayed first on the screen 
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when the user operates the auxiliary menu operation button 1380. The auxiliary menu condition 4010 includes a key- 
word button image group 4011 and a keyword change button Image 4012, on which characters 'display other words" 

are written. 

[0230] An auxiliary menu condition 4020 shows the condition of the auxiliary menu when the user positions the cursor 
s location at the keyword-written button image "preset" using the auxiliary cursor buttons 1 382 and 1 383 on the auxiliary 
menu operation button 1380 and presses the auxiliary determination button 1381. Compared with the auxiliary menu 
condition 4010, the auxiliary menu condition 4020 further includes a function button image group 4021, which is a 
group of button Images on which function names are written. 

[0231] When one button image in the function button Image group 4021 is selected by the user, the auxiliary menu 
10 controller 341 8 stores the execute control information for executing the function corresponding to the selected button 
Image in the execution instruction/apparatus condition information storage unit 1412. When one button image in the 
keyword button image group 401 1 1s selected by the user, the auxiliary menu controller 341 8 determines button Images 
related to the keyword written on the selected button Image, and updates the function button Image group 4021 to 
display the determined button images. 
IS [0232] The data that is stored in the auxiliary menu display control information storage unit 3417 will be explained 
below. 

[0233] Fig. 19 shows the data that the auxiliary menu display control Information storage unit 3417 stores. 

[0234] As shown in Fig. ig, the auxiliary menu display control information storage unit 3417 stores a plurality of 
pieces of function item Information 351 0, each of which is the information related to a function Item that can be included 

20 in the function button image group 4021. a plurality of keyword information 3520, each of which is the information 
related to a keyword that can be Included In the keyword button image group 4011 , semantic relation information 3530 
that is the information on the semantic relations between the function Items and the keywords, and function item/ 
keyword condition information 3540 that is referred to when It is judged whether a function item or a keyword should 
be displayed on the auxiliary menu. The auxiliary menu display control infomnation storage unit 3417 has an area for 

25 storing the values that are necessary for the calculation by the auxiliary menu controller 3418. 

[0235] Note that the function Item/keyword condition infonmatlon 3540 is updated by the auxiliary menu controller 
3418 according to the user operation. On the other hand, the function Item Informatbn 3510. keyword information 
3520, and the semantic relation information 3530 are stored In the auxiliary menu display control Information storage 
unit 3417 in advance, and are not updated. 

30 [0236] Each piece of the function item information 3510 includes a number, a function item name, and a function 
Item execution instruction information. One piece of function item information 3510 is Included In the auxiliary menu 
display control Information storage unit 3417 tor one function that can be selected by the user from the auxiliary menu. 
The plurality of pieces of function item Information 3510 are numbered from 0 in consecutive order 
[0237] Here, a piece of function item execution instruction information is described In the same form as a piece of 

3S execution instruction information described in the first embodiment. 

[0238] Fig. 20 is an example of the list of the numbers given to the function Item information 3510 and the corre- 
sponding function Items. 

[0239] Each piece of the keyword information 3520 includes a number and a keyword. One piece of keyword infor- 
mation 3520 is included in the auxiliary menu display control information storage unit 3417 for one keyword that can 
40 be displayed on the auxiliary menu. The plurality of pieces of keyword information 3520 are numbered in consecutive 
order starting from the number that follows the last number given to a piece of function item Information. When the 
number of the pieces of function item Information 3510 Is "N", each piece of the keyword information 3520 is given the 
number whose value is equal to or larger than "N". In addition, when the number of the keywords is "M", the value of 
the largest number given to a piece of keyword information 3520 is "N+M-1". 
[0240] In this specification, "N" is set at 47 and "M" Is set at 62. 

[0241] Fig. 21 Is an example of the list of the numbers given to the keyword informatk^n 3520 and the corresponding 

keywords. 

[0242] As shown in Fig. 21 , a keyword is a character string that defines a function from the viewpoint of the easiness 
of finding a desired function by the user. 
so [0243] In the semantic relation Information 3530, the plurality of pieces of function Item Information 3510 and keyword 
Information 3520 are represented by consecutive numbers. When one of the consecutive numbers is selected, each 
piece of function Item Information 3510 or keyword information 3520 that is directly related to the piece of function item 
information or the keyword information that corresponds to the selected number is shown in the semantic relation 
Information 3530. 

55 [0244] Note that one function item or keyword has a direct semantic relation to at least one function item or keyword. 
As a result, one keyword has a direct or indirect semantic relation to at least one function Item. An indirect semantk: 
relation between a keyword and a function Item means a semantic relation between the keyword and the function item 
via another keyword. For instance, when a first keyword has a direct semantic relations to a second keyword, and 
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when the second keyword has a direct semantic relation to a function item, the first keyword has a indirect semantic 
relation to the function item. In addition, when the second keyword has a direct semantic relation to a third keyword, 
the third keyword has an indirect relation to the function item. 

[0245] In Fig. 19, an example of the semantic relation information 3530 is shown. The number of function items is 
5 "N", that of the keywords is "M", and the function items and the keywords are represented by consecutive numbers 
from 0 to "N+M-1 When the function item or keyword of one column (column "j") has a semantic relation to the function 
item or keyword of one line (line "i"), the element that is positioned at the column "j" and line "I" Is weighted according 
to the level of the relation. In Fig. 19, the weights are all 'V. 

[0246] For instance, the example in Fig. 19 shows that the keyword represented by the number "N+M-1" has a 

10 relation to the function item represented by the number 3. 

[0247] Note that the semantic relation information 3530 shows the relations between focused keywords or function 
items on the primary side and keywords or function items on the secondary side to which the keywords or function 
items on the primary side have semantic relations. As a result, even if a keyword "preset" on the primary side has a 
semantic relation to a function item "setting image quality in recording" on the secondary side, a function item "setting 

IS image quality in recording" on the primary side cannot have a semantic relation to a keyword "preset" on the secondary 
side. 

(Operation) 

20 [0248] The operation by the digital broadcast receiving system 3000, the structure of which has been described, will 
be explained below. 

[0249] The recording and replaying apparatus 31 00 records and replays digital broadcast programs under the control 

of the controller 3110. 

[0250] The operation by the digital broadcast receiving apparatus 3100 in recording and replaying is basically the 
25 same as the recording and replaying apparatus 1100 that has been described in the first embodiment. 

[0251] The controller 3110 displays a standard menu and a following screen for user input to receive user instruction, 
and executes a function such as recording and replaying according to the user instruction. In addition, when one of 
the auxiliary determination button 1 381 and the auxiliary cursor buttons 1 382 and 1 383 on the auxiliary menu operation 
button 1380 is pressed by the user, the controller 3110 displays the auxiliary menu to provide a user interface for 
30 enabling the user to select a desired function apart from a standard menu in a hierarchical structure. 

[0252] The operation by the digital broadcast receiving apparatus 3100 will be described In detail, focusing on the 
operation by the auxiliary menu controller 3418, which controls the update and the renewal of the auxiliary menu 
according to user operation of the auxiliary menu operation button 1380. 
[0253] Fig. 22 is a flowchart showing the operation by the auxiliary menu controller 3418. 
35 [0254] When informed that the user has operated the auxiliary menu operation button 1 380 on the remote controller 
1 300, the auxiliary menu controller 341 8 judges whether the auxiliary menu is displayed to know whether Initialization 
is necessary (step S401). 

[0255] More specifically, when no auxiliary menu is displayed and when any button on the auxiliary menu operation 
button 1380 is pressed by the user, the auxiliary menu controller 3418 judges that the user operation as the first op- 
40 eration on the auxiliary menu (step S401), and initializes the function item/keyword condition information 3540 in the 
auxiliary menu display control information storage unit 3417 (step S402). 

[0256] Here, the update of the function item/keyword condition information 3540 will be explained. 

[0257] Fig. 23 shows the principle of the updating operation of the function item/keyword condition information 3540. 

[0258] Nodes 301 , 302, 303, 304, .... and 311 are equivalent to variables in a computer program and correspond to 
45 the function itenrVkeyword condition information 3540. 

[0259] The arrows pointing the nodes 301 and the like represent the input of the evaluation values to the nodes 301 

and the like. Values l[0], f [1 ], . . . , and l[N+M-1 ] represent predetermined values that are input according to user operation. 

[0260] That the outputs V[0], V[1], .... V[N+M-1] from the nodes 301 and the like are inputs of the nodes 301 and the 

like are shown by the marks of solidly shaded boxes at the points of intersections of two lines. The solidly shaded box 
50 marks correspond to the content of the semantic relation Information 3530. The auxiliary menu controller 341 8 updates 

the function item/keyword condition Information 3540 by referring to the semantic relation information 3530. Here, the 

input operation of the output values from the nodes 301 and the like is performed in accordance with a predetermined 

control procedure, and is not infinitely repeated. 

[0261] Note that the larger evaluation value a node holds, the more highly the user is interested in the function item 
55 or keyword corresponding to the node. Here, that the user is interested in a function item or keyword means that the 
function item or keyword has a relation to the function item that the user wants to execute. 

[0262] Suppose that the element positioned at the column "j" and line "i" is represented by "T[i][]]". input Vin[i] Into 
the "i"th node is expressed by an expression 1 described below (refer to Fig. 23). 
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Vln[i] = Z(T[I] [jlV[j])+l[i] (Z is the sum of the "T[i][j]VO]" when -j" is "0' to "N+M-1 ■) Expression 1 

[0263] In this specification, the nodes are numbered, for instance the node 301 is numbered as the 0th node and 
s the node 302 as the first node. Note that the "i"th node corresponds to a piece of function item information or keyword 
information that is given number "i". 

[0264] When the user selects a keyword and the 'ilh node corresponds to the selected keyword, the auxiliary menu 
controller 3418 sets the evaluation value l[i] as a predetermined positive value. When the user presses the keyword 
change button image 4012 "display other words", the auxiliary menu controller 3418 sets the evaluation values l[i]s 
10 corresponding to the keywords and function items that are displayed as predetermined negative values. 

[0265] The relation between a value input into a node and the value output from the node after the value input is 
expressed by an expression 2 for the 0th to N-1th nodes and by an expression 3 for the Nth to N+M-1 th nodes. 

,5 V[l] = OUT_LEVEL1 (when Vin[l]>LEVEL1 ) 

V[i] = 0 (when Vln[i]^LEVEL1 ) Expression 2 




20 V[l] = OUT.LEVELI +l[i] (when Vin[l]>LEVEL2) 

V[i] = 0 (when Vin[i]^LEVEL2) Expression 3 

[0266] Note that the "OUT^LE VEL1 " is a predetermined value and the "LEVEL1 " and "LEVEL2" are predetermined 
25 thresholds in the Expression 2 and 3. 

[0267] When the user selects one keyword, a positive evaluation value is input Into the node corresponding to the 
selected keyword, and later the output of the node increases the degree of the input value into the node corresponding 
to a function item or keyword in the semantic relation information 3530. 

[0268] When the degree of the input value into a node is increased and the value is larger than a threshold, the 
30 degree of the output is increased. In addition, when it is controlled that the output of increased degree is again given 
as an Input, the degree of the Input value of the node that has a semantic relation to the first node. 
[0269] As a result, when a keyword is selected, the increase of the degree of the input value into the function item 
related to the keyword spreads from a function item that has the closest semantic relation to the selected keyword to 
a function item that has the furthest semantic relation by repeating the control that has the output from a node input 
35 into the node. 

[0270] Incidentally, the expansion of the input value increase can be similar to the connection procedure of conception 

according to the spreading activation theory of semantic processing in psychology The spreading activation theory is 
described In, for instance, "Cognitive Psychology Critical Study Collection Vol. 2, Memory Cognition Critical Study 2-4. 
"Spreading Activation Theory of Semantic Processing" (Sel-Shin Publishing, pp.64-67)". 
40 [0271] The process of the flowchart in Fig. 22 will be again explained below. 

[0272] The auxiliary menu controller 3418 Initializes the evaluation values held by the nodes, i.e., the function item/ 
keyword condition information 3540 (step S402). 

[0273] After the Initialization, the auxiliary menu controller 341 8 performs a display keyword determination processing 
for obtaining the keywords to be displayed on the auxiliary menu (step 8403). The display keyword determination 
45 processing will be described later. 

[0274] After the display keyword determination processing, the auxiliary menu controller 341 8 displays the auxiliary 
menu including a plurality of button images on which the determined keywords are written and a button Image on which 
the character string "display other words" is written on the monitor 1200 via the auxiliary menu display unit 3420 (step 
S404). 

50 [0275] As a result, the auxiliary menu in the auxiliary menu condition 4010 shown in Fig. 18 Is displayed on a part 
of the monitor 1200. 

[0276] When the button image "display other words" is selected by the user (step S405), the auxiliary menu controller 
341 8 inputs negative evaluation values Into the nodes corresponding to the function items and keywords that are now 
displayed to update the values held by all the nodes (step S406), and performs a display keyword determination 
55 processing (step 8403). 

[0277] Meanwhile, when a keyword-written button Image Is selected by the user (step 8407), the auxiliary menu 
controller 3418 inputs a positive evaluation value into the node corresponding to the selected keyword to update the 
values held by alt the nodes (step 8408), and determines the function items to be displayed on the auxiliary menu (step 
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S409). 

[0278] The function items to be displayed on the auxiliary menu are detemnined by selecting six nodes in decreasing 
order of evaluation value out of the nodes corresponding to function Items and setting the f unctbn items corresponding 

to the selected six nodes as the function items to be displayed. 

[0279] After determining the function items to be displayed on the auxiliary menu (step S409). the auxiliary menu 
controller 3418 performs a display keyword determination processing (step S403). 

[0280] Meanwhile, when a function item is selected by the user, the auxiliary menu controller 3418 refers to the 
function item information in the piece of function item information 3510 corresponding to the selected function item, 

and stores the execute control Information In the execution Instruction/apparatus condition information storage unit 
1412 to instruct the execution of the function corresponding to the selected function (step S410). The control related 
to the auxiliary menu is completed with step S410. 

[0281] Note that when the execute control information Is stored in the execution instruction/apparatus condition in- 
formation storage unit 1412, the function execution controller 1 41 3 executes a function according to the stored execute 

control information as has been described in the first embodiment. 

[0282] The display keyword determination processing (step S403) will be explained in detail with reference to Fig. 24. 
[0283] Fig. 24 is a flowchart showing the display keyword determination processing by the auxiliary menu controller 
3418. 

[0284] The auxiliary menu controller 3418 sets a variable "k" that is the order of display of keyword as "1" (step 

5501 ) , and refers to the evaluation values of the nodes, i.e. , the function item/keyword condition information 3540. The 
auxiliary menu controller 3418 obtains the keyword that has semantic relations to the function items the number of 
which is closest to the half of the number of the function items whose evaluation values are larger than the threshold 
"LEVEL1 and determines the obtained keyword as the keyword to be displayed at the top on the auxiliary menu (step 

5502) . 

[0285] Here, a keyword that has a semantic relation to a function item means that the output of the node correspond- 
ing to the keyword is the Input into the function item (refer to Figs. 19 and 23). 

[0286] After obtaining the keyword displayed at the top, the 3418 auxiliary menu controller excludes the function 
Items to which the keyword has semantic relations from the objects of calculation (step S503). Here, the calculation 
objects are the objects of the calculation for obtaining the keywords at step S506. 

[0287] After the step S503, the auxiliary menu controller 341 8 Increases the value of the variable "k" by 1 (step S504). 
and judges whether the number of the function items that are now the calculation objects is at least one and the value 
of the variable "k" is less than 5 (step S505). 

[0288] When the result of the judgement at step S505 is affirmative, the auxiliary menu controller 3418 obtains one 

keyword that has semantic relations to the largest number of function items that are now the calculation objects, and 
determines the keyword as the keyword displayed "k"th on the auxiliary menu (step S506). The process returns to step 
S503, at which the auxiliary menu controller 3418 excludes the function items to which the keyword has semantic 
relations from the calculation items. 

[0289] As a result, until the judgement result at step S505 is negative, the operations at steps S506, S503, and S504 

are repeated. The number of obtained keywords correspond to the number of the repetition of the processes. 
[0290] When the judgement result at step S505 Is negative, the auxiliary menu controller 3418 obtains one keyword 
that is not the keyword displayed at the top and has semantic relations to the largest number of function items to which 
the top-displayed keyword has semantic relations, and determined the obtained keyword as the keyword that is dis- 
played at the bottom on the auxiliary menu (step S507). 

[0291] As has been described the auxiliary menu controller 3418 determines the keywords that are to be displayed 
on the auxiliary menu. 

[0292] The operation by the auxiliary menu controller 3418 will be concretely explained with an example of user 
operation. 

[0293] When no auxiliary menu is displayed and when the user presses one of the auxiliary determination button 
1 381 and the auxiliary cursor buttons 1 382 and 1 383 on the auxiliary menu operation button 1 380, the remote controller 
transmits a signal informing the controller 311 0 of the user operation. The auxiliary menu operation receiving unit 341 9 
in the controller 3110 detects the transmitted signal, and informs the auxiliary menu controller 3418 of the signal. 
[0294] On receiving the signal, the auxiliary menu controller 3418 displays the auxiliary menu in the auxiliary menu 
condition 4010 shown in Fig. 18 on a part of the monitor 1200 via the auxiliary menu display unit 3420 (steps S401, 
S402, S403, and S404). 

[0295] IVlore specifically the auxiliary menu display unit 3420 transfers the image to be displayed to the output unit 
1170 using the image data stored In the image data storage unit 3414. On receiving the image to be displayed, the 
output unit 1170 displays the auxiliary menu in the auxiliary menu condition 4010 on a part of the monitor 1200, 
[0296] Note that on the other part of the monitor 1 200 apart from the part on which the auxiliary menu is displayed, 
the image that has been displayed on the monitor 1200 is kept displayed. For instance, when the user has been 
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operating a standard menu, the standard menu is kept displayed. 

[0297] In this condition, whenever the user presses the auxiliary cursor button 1383, the auxiliary menu controller 
3418 moves the cursor on the monitor 1 200 so that the cursor would be positioned at a button image in rotation in the 
order of "recording", "setting system", "condition confirmation", "breakdown", "worry and trouble", "preset", and "display 

5 other words". When the user presses the auxiliary cursor button 1382, the auxiliary menu controller 3418 moves the 
cursor so that the cursor would be positioned at a button image in rotation in the inverse order. 
[0298] Note that the user operates the auxiliary menu using the auxiliary menu operation button 1 380 on the remote 
controller 1 300, so that the user may perform another operation with the operation of the auxiliary menu by pressing 
another button apart from the auxiliary menu operation button 1380 on the remote controller 1 300. 

10 [0299] When judging that the keyword with which a desired function is connected is not displayed on the auxiliary 
menu, the user selects "display other words". As a result, the auxiliary menu controller 3418 updates the keywords on 
the auxiliary menu (steps S401, S405, S406, S403. and S404). 

[0300] i When the auxiliary menu is in the auxiliary menu condition 4010, when the user positions the cursor not at 

the button image "display other words" but at the button image "preset", and when the user presses the auxiliary 
?5 determination button 1381 on the remote controller 1300, the auxiliary menu controller 3418 performs operations in 
the order of steps S401 , S405, S407, S408, S409, S403, S404 (refer to Fig. 22). 

[0301] More specifically, the auxiliary menu controller 3418 inputs a predetermined positive value into the node cor- 
responding to the selected keyword "preset" to perform the calculations shown by the Expressions 1 to 3. As a result, 
the evaluation value held by the function item or keyword to which the keyword "preset" has a semantic relation is 

20 increased (step S408). After step S408, the auxiliary menu controller 3418 selects six nodes in decreasing order of 
evaluation value out of the nodes whose evaluation values have been increased, and determines the function items 
corresponding to the selected nodes as the function items to be displayed (step S409). Then, the auxiliary menu 
controller 3418 performs a display keyword determination processing (step S403, refer to Fig. 24). updates the auxiliary 
menu via the auxiliary menu display unit 3420 to display the determined keywords and function items that have been 

25 determined on referring to the function item information 3510 and the keyword information 3520 (step S404). 
[0302] As a result, the displayed auxiliary menu Is In the auxiliary menu condition 4020. 

[0303] In this condition, whenever the user presses the auxiliary cursor button 1 383 on the renfK)te controller 1 300, 
the auxiliary menu controller 3418 moves the cursor on the monitor 1200 so that the cursor would be positioned at a 
button image in rotation in the order of "preset for recording by genre", "preset cancellation", "preset condition confir- 

30 mation", "preset for recording programs In program table", "disk condition confirmation", "setting image quality in re- 
cording", each of the keyword button image group 4011 , and "display other words". Note that the cursor moves In the 
direction from the top to bottom for the keyword button image group 4011 . When the user presses the auxiliary cursor 
button 1 382, the auxiliary menu controller 3418 moves the cursor so that the cursor would be positioned at a button 
image in rotation in the inverse order 

35 [0304] In this condition, when the user selects one of the button images in the function button image group 4021, 
the auxiliary menu controller 3418 refers to the function item information 3510, and stores the execute control infor- 
mation for executing the function item corresponding to the selected button Image In the execution Instruction/apparatus 
condition information storage unit 1412 (step S410). Then, the auxiliary menu controller 3418 sets the auxiliary menu 
in non-display condition. With this operation, the control related to the auxiliary menu is completed. 

40 [0305] Meanwhile, when the auxiliary menu in the auxiliary menu condition 4020 and the user selects a button image 
in the keyword button Image group 4011, for Instance, the button image "detect", the auxiliary menu controller 3418 
performs the operations in the order of steps S401 , S405, S407, S408, S409. S403, and S404 once more. 
[0306] As has been described, when the user selects a keyword at several times, all the user selections are used in 
the determination of the function items to be displayed on the auxiliary menu. More specifically. It is not that the user 

^ directly designates a function operation but that the user selects a keyword and the recording and replaying apparatus 
1 1 00 specifies function operations that are supposed to be desired by the user according to the selected keyword and 
displays the function button Image group 4021 . In other words, the recording and replaying apparatus 1100 predicts 
function operations that are supposed to be desired by the user and provide a user interface. 

50 (Third Embodiment) 

[0307] The user Interface that displays buttons not necessarily selected, i.e.. has the recommend display appear as 
the auxiliary menu in the first embodiment realizes an apparatus with a variety of user-convenient and self-explanatory 
functions. In the third embodiment, an example in which a function that may not be included in an ordinary menu 
55 structure is shown by a recommend display when necessary will be explained. The recommend display in this speci- 
fication means to specify and display a function operation that has not been directly designated by the user, i.e., to 
predict and display a function operation that is supposed to be desired by the user 

[0308] A user Interface apparatus according to the third embodiment of the present invention will be explained below 
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with reference to Figs. 25 to 28. 
(Structure) 

6 [0309] Fig. 25 shows a structure of a digital broadcast receiving system 5000 equipped with a user interface apparatus 
according to the third embodiment of the present invention. 

[0310] The digital broadcast receiving system 5000 includes a digital broadcast receiving apparatus 5100, a monitor 
5200 for displaying images and outputting audio, and a remote controller 5300 for receiving user Instructions and 
transmitting the received Instructions to the digital broadcast receiving apparatus 5100. 

10 [0311] The digital broadcast receiving apparatus 5100 includes an infrared ray reception unit. The digital broadcast 
receiving apparatus 5100 detects a signal that has been transmitted from the remote controller 5300 according to a 
user instruction using the infrared ray reception unit, executes such functions such as receiving a digital broadcast 
program and outputting a received program In response to the user Instruction, and outputs an image for displaying a 
button for function execution Instruction on the monitor 5200. The digital broadcast receiving apparatus 5100 Includes 

'5 a broadcast receiving antenna 5101 , a receiving unit 5110, a recording unit 5120, a replaying unit 6130, an output unit 
5140. a controller 5150, and a timer 5160 that has time measuring function. 

[0312] Here, the controller 5150 includes a CPU and a storage unit such as a memory. The controller 5150 realizes 
a execution control function for controlling the operations by each elements in the digital broadcast receiving apparatus 
5100 according to the signals transmitted from the remote controller 5300 and time infornnation transferred from the 
20 timer 5160. The storage unit stores a program for the control. 

[0313] More specifically, the controller 5150 receives the Information on a program table from the Information that 
the receiving unit 5110 has received, has the receiving unit 5110 receive a program on a channel that the user has 
designated, and the recording of the history of watched programs. Here, the program table is the same as explained 
in the first embodiment. 

25 [0314] The controller 5150 realizes the user interface control function, and outputs a graphical user Interface image 
such as a button image for having the user designate a function execution to the monitor 5200. 
[031 5] The receiving unit 511 0 includes a tuner, a system stream decoder of MPEG. The receiving unit 511 0 receives 
digital broadcast programs via the broadcast receiving antenna 5101, demodulates the received programs, and sep- 
arates a program that the user desires into packets. In addition, the receiving unit 5110 transfers the separated program 

30 to the replaying unit 5130 or the recording unit 5120, and receives a program table as well as programs. 

[0316] The recording unit 5120 includes a hard disk or a DVD-RAM. The recording unit 5120 records the programs 
data that has been transferred from the receiving unit 5110 on the hard disk, the DVD-RAM, or the like. 
[0317] The replaying unit 51 30 includes a decoder for decoding images and audio of MPEG, expands the program 
data transferred from the receiving unit 5110, and transfers data for displaying images and outputting audio to the 

35 output unit 5140. 

[031 8] On receiving the data for displaying an image from the replaying unit 51 30 and the controller 51 50, the output 
unit 5140 combines the data Into a TV signal as necessary, and output the TV signal to the monitor 5200 as well as 
the data for outputting audio that has been transferred from the replaying unit 5130. 

[0319] The remote controller 5300 includes a determination button, a cursor button, and the like for the operation of 
40 the graphical user interface on the monitor 5200. When any of the buttons is pressed by the user, the remote controller 
5300 transmits a signal for identifying the pressed button to the digital broadcast receiving apparatus 5100. Here, the 
determination button Is used when pressing a button Image displayed on the monitor 5200. 

[0320] The functional structure of the controller 51 50. which realizes the execute control function and the user inter- 
face control function, will be explained below. 
45 [0321] Fig. 26 is a functional block diagram of the controller 51 50. 

[0322] The execute control function and the user interface control function of the controller 5150 are realized by a 
function controller 51 51 . a user input receiving unit 51 52, an apparatus elements controller 51 53, an apparatus condition 
information storage unit 51 54, a time management unit 5155, a GUI controller 5156, and a program Information storage 
unit 5157. 

so [0323] The function controller 51 51 has the apparatus elements controller 51 53 controls each elements of the digital 
broadcast receiving apparatus 5100 such as the receiving unit 5110, the replaying unit 5130, and the like to have a 
program table be fetched every hour and to have the function corresponding to a user instruction be executed. The 
function controller 5151 refers to the apparatus condition information storage unit 5154 and the program information 
storage unit 5157, has the GUI controller 5156 display a button image as necessary using the time management unit 

55 51 55. For these operations, the function controller 51 51 decodes a user instruction that has been received by the user 
Input receiving unit 51 52, and gives an instruction to the apparatus elements controller 51 53 and the like according to 
the received user instruction. 

[0324] A program table that has been fetched by the function controller 5151 is stored in the program information 
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storage unit 51 57. Note that when fetching a new program table, the function controller 51 51 abandons the program 
table that has been fetched. In addition, the function controller 5151 detects a program that satisfies a predetermined 
condition from a program table. 

[0325] The GUI controller 51 56 receives an instruction from the function controller 5151 , creates a button image that 
displays a designated character string, and displays the created button image on the monitor 5200 via the output unit 
5140. 

[0326] The user input receiving unit 5152 detects a signal transmitted from the remote controller 5300 to receive 
user operation, and transfers the received user operation to the function controller 5151. When the GUI controller 5 1 56 
displays a button image on the monitor 5200, the user input receiving unit 5152 receives user operation such as the 

designation of a button image and cursor move. 

[0327] On receiving an instruction from the function controller 5 1 51 , the apparatus elements controller 51 53 controls 
the receiving unit 5110, the replaying unit 51 30, and the like to execute a variety of operations. The apparatus elements 
controller 51 53 stores the executing states of the operations and the conditions of each of the elements In the controller 

5150 in the apparatus condition information storage unit 5154. As a result, the information on the executing conditions 
of each of the elements such as the channel number of a program that is now being received is stored in the apparatus 
condition information storage unit 5154. 

[0328] The time management unit 5155 refers to the time Information transferred from the timer 5160, notifies the 
function controller 5151 of the elapse of a designated period of time, and transfers the time information to the function 
controller 5151. 

(Operation) 

[0329] The operation by the digital broadcast receiving system 5000, the structure of which has been described, will 

be explained below. 

25 [0330] The digital broadcast receiving apparatus 51 00 executes the reception of digital broadcast programs and the 
like under the control of the controller 5150. 

[0331] A program is received by the digital broadcast receiving apparatus 5100 in the manner described below A 
radio wave over which multiplexed information such as a program is broadcasted is received by the receiving unit 511 0 
via the broadcast receiving antenna 5101, and the information on a desired program is extracted. The graphic and 
30 audio data of the desired program is expanded by the replaying unit 51 30, and the information is input to the monitor 
5200 via the output unit 5140. 

[0332] The controller 5150 controls the digital broadcast receiving apparatus 5100 according to user operation to 
enable the reception of a program on a channel that has been designated by the user. In addition, the controller 5150 
realizes the function described below. When the user switches channels during the commercials in a program and 
35 watches another program, the controller 5150 displays a button image on the monitor 5200 when one minute has 
elapsed since the channel was switched so that the user would switch channels to watch the originally watched program. 
A program switch operation responding processing, a recommend display purpose timer time elapse processing, and 
a watching determination detection purpose timer time elapse processing, which each are performed by the function 
controller 5151 , will be described below. 
40 [0333] Fig. 27 is a flowchart showing the program switch operation responding processing, the recommend display 
purpose timer time elapse processing, and the watching determination detection purpose timer time elapse processing, 
which each are performed by the function controller 5151 . 

[0334] These processings mean that the function controller 5151 designates a period of time and has the time man- 
agement unit 5155 notifies itself of the elapse of the designated period of time, i.e., the function controller 5151 uses 
45 the time management unit 5155 as a logical timer. The program switch operation responding processing is performed 
when the function controller 5151 detects that the user has switched channels. The program switch operation respond- 
ing processing includes the procedure for setting two logical timers, i.e.. a recommend display purpose timer and a 
watching determination detection purpose timer. 

[0335] Meanwhile, the recommend display purpose timer time elapse processing is performed by the function con- 
50 troller 5151 that is informed of the elapse of the period of time by the time management unit 5155 when a period of 
time for which the recommend display purpose timer has been set is elapsed. The watching determinatkm detection 
purpose timer time elapse processing is performed by the function controller 5151 that is Informed of the elapse of the 
period of time by the time management unit 5155 when a period of time for which the watching determination detection 
purpose timer has been set is elapsed. 
55 [0336] When detecting the channel switch by the user, the function controller 5151 stops the watching determinatbn 
detection purpose timer (step S5501 ), and judges whether a watching determination flag is 'ON' (step S5502). Here, 
the watching determination flag is one bit of data. The watching determination flag is set as "ON" when the user has 
been watching a program on a channel for ten minutes and a watching condition is determined, and is set as "OFF" 
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when the channel is switched. 

[0337] At a judgement step S5502, when the watching determination flag is "ON", the function controller 5151 sets 
the watching determination flag as "OFF"! (step S5503), and determines the content of a button Image that Is to be 
displayed on the monitor 5200 (step S5504), The function controller 5151 sets the recommend display purpose timer 

s for one minute and starts the recommend display purpose timer (step S5505), and sets the watching determination 
detection purpose timer for 10 minutes and starts the watching determination detection purpose timer (step S5506). 
The button Image content that Is to be determined Includes the title of the program that the user had been watching 
for at least 10 minutes and the reason why the button image is displayed. The title of the previously watched program 
is held at step S5523, which will be described later. 

10 [0338] For instance, when the user has been watching a movie "XX" for at least 10 minutes and switches channels 
during the commercials, the content of the button image is character string "return to movie "XX" (one minute has 
elapsed)'. The button Image content shows that the button Image is a graphical user Interface for reminding the user 
that he/she had been watching the movie "XX" because one minute has elapsed since the channel switch and the user 
can forget having been watching the movie "XX". 

J5 [0339] Meanwhile, at the judgement step S5502, when the watching determination flag Is "OFF", the function con- 
troller 5151 skips steps S5503 to S5505 and sets the watching determination detection purpose timer for 10 minutes 
and starts the watching determination detection purpose timer (step S5506). 

[0340] After the set of the watching determination detection purpose timer, the function controller 5151 responds to 
the channel switch by the user, instructs the apparatus elements controller 51 53 to have the receiving unit 5110 switch 
20 channels to receive another program (step S5506). The processing responding to the channel switch by the user is 
completed with step S5506. Note that when the user switches channels again, the function controller 5151 performs 
the program switch operation responding processing once more. 

[0341] Then, when one minute has elapsed since the set of the recommend display purpose timer at step S5505, 
the function controller 5151 is informed of the time elapse by the time management unit 5155, and performs the rec- 
2S ommend display purpose timer time elapse processing. The function controller 5151 has the GUI controller 51 56 create 
the button image that displays the character string "return to movie "XX" (one minute has elapsed)", and has the monitor 
5200 display the created button image via the output unit 5140, I.e., the recommend display appears on the monitor 
5200 (step S5511), 

[0342] Meanwhile, when 1 0 minutes has elapsed since the set of the watching determination detection purpose timer 
30 at step as 5506, the function controller 51 51 is informed of the time elapse by the time management unit 5155, and 
performs the watching determination detection purpose timer time elapse processing. More specifically, when the rec- 
ommend display that has been described appears, the function controller 5151 cancels the recommend display (step 
S5521 ), sets the watching determination flag as "ON" (step S5522), and holds the information on the program that the 
user has been watching for 10 minutes, or the program for which the user watching is determined (step S5523). Here, 
35 the program for which the user watching is determined is the program that the receiving unit 5110 is now receiving. To 
hold the information on the program means that the function controller 5151 refers to the information on the channel 
of the presently received program stored in the apparatus condition infornnatlon storage unit 51 54 and to the program 
table stored in the program information storage unit 5157, and obtains and holds the program title. 
[0343] As a result of these operations mainly performed by the function controller 5151, for instance, when the user 
40 switches channels after having been watching the movie "XX" for at least 1 0 minutes to start watching a live coverage 
baseball game program, a button image Is displayed on the monitor 5200 as shown in Fig. 28 when one minute has 
elapsed since the user started watching the baseball game program. 

[0344] Fig. 28 shows a monitor on which a recommend display appears for recommending watching the previously 

watched program. 

45 [0345] As shown in Fig. 28, when the button image including the character string "return to movie "XX" (one minute 
has elapsed)" is displayed, and when the user presses the determination button on the remote controller 5300. the 
digital broadcast receiving apparatus 5100 performs the same operation as when the user designates the channel of 

the movie "XX" and switches channels. 

[0346] Note that the recommend display purpose timer is set for one minute because it is supposed to take about 
50 one minute for the user to complete zapping, i.e., to switch channels and browse other programs during one program 
or the commercials In the program. The watching determination detection purpose timer is set for 10 minutes because 
it is supposed that the user wants to continue to watch a program that had been watched for at least 10 minutes. 

(Fourth Embodiment) 

55 

[0347] In the fourth embodiment, a user interface apparatus that has a recommend display appear according to user 
information such as timetable and taste will be explained. 

[0348] The user interface apparatus according to the fourth embodiment of the present invention will be explained 
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below with reference to Figs. 29 to 35. 
(Structure) 

s [0349] The structure of a digital broadcast receiving system including the user interface according to the fourth enn- 
bodiment of the present invention is almost the same as thai of the digital broadcast receiving system 5000 according 
to the third embodiment (refer to Fig. 25). The functional structure of a controller in a digital broadcast receiving appa- 
ratus in the digital broadcast receiving system according to the fourth embodiment, however, differs from that of the 
controller 5150 in the third embodiment (refer to Fig. 26). 

10 [0350] Fig. 29 is a functional block diagram of a controller 6150 in the digital broadcast receiving apparatus In the 
fourth embodiment. 

[0351] The controller 6150 performs the execute control function and the user Interface control function as the con- 
troller 5150 In the third embodiment. The controller 61 50 Includes a function controller 61 51, a user input receiving unit 
5152. an apparatus elements controller 5153, an apparatus condition information storage unit 5154, a time manage- 
rs ment unit 5155, a GUI controller 5156, a program information storage unit 5157, and a user information storage unit 
6158. Note that the elements In Fig. 29 and in the controller 5150 in the third embodiment that are given the same 
numerical numbers have the same functions. 

[0352] The function controller 6151 has the apparatus elements controller 5153 control the elements in the digital 
broadcast receiving apparatus such as the receiving unit 5110 and the replaying unit 51 30 to have a program table be 

20 fetched every hour and to have the function corresponding to a user Instruction be executed. The function controller 
6151 refers to the apparatus condition information storage unit 5154 and the program information storage unit 5157. 
has the QUI controller 5156 display a button image as necessary using the time management unit 5155. For these 
operations, the function controller 6151 decodes a user instruction that has been received by the user input receiving 
unit 5152, and gives an instruction to the apparatus elements controller 5153 and the like according to the received 

2S user instruction. Here, a program table that has been fetched by the function controller 6161 is stored In the program 
Information storage unit 5157 as In the third embodiment. 

[0353] The functions of the function controller 6151 and the function controller 51 51 are basically the same, however, 
the function controller 6151 differs from the function controller 5151 in controlling so that the user information would 
be referred, an operation that is supposed to be desired by the user would be predicted, and a user interface with 

30 which the user instructs an operation would be provided when necessary. 

[0354] The user information storage unit 6158 stores the user Information described below. 

[0355] Fig. 30 shows the content of the user information stored in the user information storage unit 6158. 

[0356] User information 6400 includes information peculiar to the user, i.e., information on the user timetable and 

taste such as a taste genre 6401 , a favorite program 6402, a program watching history 6403, and a bedtime 6404. 

35 [0357] The taste genre 6401 shows 10 or less user favorite program genres such as sport and news. The favorite 
program 6402 shows 1 0 or less user favorite programs, and Includes the information on the titles, genres, players, and 
channels. The program watching history 6403 shows 10 or less programs that the user has watched, and includes the 
information on the titles, genres, players, and channels. 

[0358] The bedtime 6404 shows the bedtime of the user in the usual timetable. 
40 [0359] Note that the taste genre 6401, the favorite program 6402, and the bedtime 6404 are directly input by the 
user using a remote controller and the like. The program watching history 6403 is obtained by the function controller 
61 51 on referring to program tables in order toobtain the information on the programs that have been watched according 
to user operations. 

<5 (Operation) 

[0360] The digital broadcast receiving apparatus executes the reception of digital broadcast programs and the like 

under the control of the controller 6150. Here, two functions specific to the digital broadcast receiving apparatus will 

be explained. The two specific functions are a program recommendation function for recommending the user to watch 
50 a favorite program, and a bedtime responding function for recommending the user to record a presently watched 

program when the program is scheduled to end after the user bedtime. 

[0361] The processing in the program recommendation function will be first explained. 

[0362] Fig. 31 is a flowchart showing the processing in the program recommendation function. 

[0363] As shown in Fig. 31 , when detecting the watching determination of one program, i.e., detecting that the user 
55 have been watching one program for at least 10 minutes (step S6501), the function controller 6151 checks whether a 

program to be recommended Is to be broadcasted by the end of the presently watched program (step S6502). 

[0364] More specifically, the function controller 6151 obtains the present time from the time management unit 5155 

and refers to the program table stored in the program information storage unit 51 57 to obtain the time when the presently 
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received program ends. The function controller 6151 detects the program table to check whether a recommendable 
program is broadcasted between the present time and the time of the presently received program using the favorite 
program 6402 in the user information 6400 and the program titles, players, and the like in the program watching history 

6403. 

[0365] When no recommendable program is to be broadcasted (step S6503), the program recommendation function 
is completed. On the other hand, when a recommendable program is to be broadcasted (step S6503), a character 
string that is the content of a button image displayed on the monitor as a recommend display is determined (step 
S6504). The character string includes the title of the recommendable program and the reason why the program is 
recommended. When the recommendable program is detected using the taste genre 6401 , the reason is the genre of 
the recommendable program. Meanwhile, when the program is detected using the information on the players, the 
reason is the name of a player. 

[0366] For instance, when a recommendable program that has been detected using the name of a player "YZ", who 

appears in a program included in the favorite program 6402, is a drama "XX" at step S6502, the character string that 
is the content of the button image is "switch to drama "XX" ("Y2" appearing)". 

[0367] After the determination of the character string that is the content of the button image, the function controller 
61 51 refers to the program table to obtain the time the recommendable program starts, and sets the time management 
unit 51 55 for the time (step S6505). When the set time has arrived, the time management unit 51 55 notifies the function 
controller 6151 of the time arrival. When receiving the notification (step S6506), the function controller 6151 has the 
recommend display appears on the monitor (step S6507). The program recommendation function is completed with 
step S6507. 

[0368] For instance, when the start time of the drama "XX" has arrived during watching a live coverage baseball 
game program, the button image as shown in Fig. 32 appears on the monitor by the recommend display at step S6507. 
[0369] Fig. 32 shows a monitor on which a recommend display appears for advising the user to switch channels to 

tune to a recommendable program. 

[0370] When the user presses the button image using the remote controller and the like, the function controller 6151 
controls the receiving unit in the digital broadcast receiving apparatus via the apparatus elements controller 5153 to 
have the program be received. 

[0371] As a result, during watching a live coverage baseball game program, the user Is informed of the start of the 
drama "XX" in which "YZ" appears, who is a player in a program included in the favorite program 6402. Only by pressing 
the button image using the determination button on the remote controller, the user may watch the drama. 
[0372] Then, the processing in the bedtime responding functbn will be explained. 
[0373] Fig. 33 is a flowchart showing the processing in the bedtime responding function. 

[0374] As shown in Fig. 33, when detecting the watching determination of one program, i.e., detecting that the user 
has been watching one program for at least 10 minutes (step S6601), the function controller 6151 obtains the time 
when the program ends on referring to the program table, and compares the end time with the bedtime 6404 in the 
user information 6400 (step S6602). 

[0375] As a result of the comparison at step S6602, when the end time of the program is later than the bedtime 6404 
(step S6603), character string that is the content of a button image displayed on the monitor as a recommend display 
is determined (step S6604), and the recommend display appears on the monitor (step S6605), The bedtime responding 
function Is completed with step S6605. 

[0376] As a result, the button image as shown in Fig. 34 is displayed on the monitor, for instance. 

[0377] Fig. 34 shows a monitor on which a recommend display appears for advising the user to record the presently 

watched program. 

[0378] Fig. 34 shows an example in which the bedtime 6404 of the user is 1 :00 am. (refer to Fig. 30), the user started 
watching a live coverage baseball game program at 0:1 0 am., and the present time is 0:20 am. A button Image showing 
the information that the baseball game program ends at 2:00 am. with the recommendation to record the program is 

displayed on the monitor. 

[0379] When the user presses the button image using the remote controller and the like, the function controller 6151 
controls the recording unit in the digital broadcast receiving apparatus via the apparatus elements controller 5153, and 
has the recording unit record the presently watched program. 

[0380] As a result, the user is informed that the end time of the presently watched program is later than the usual 
bedtime, and presets the digital broadcast receiving apparatus to record the program only by pressing the button Image 
using the determination button on the remote controller. 

(Other Examples) 

[0381] Fig. 35 shows a structure of a digital broadcast receiving system 7000 provided with a digital broadcast re- 
ceiving apparatus 7100, which is another example of the digital broadcast receiving apparatus according to the fourth 
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embodiment. 

[0382] The digital broadcast receiving apparatus 7100 includes a broadcast receiving antenna 5101 . receiving units 
51 10 and 7110, a recording unit 5120, a replaying unit 5130, an output unit 5140, and a timer 5160, a recorder 7170, 

and a controller 7150. The digital broadcast receiving apparatus 7100 completely differs from the digital broadcast 
s receiving apparatus 5100 in Fig. 25 in including two receiving units and the recorder 7170. 

[0383] As a result, it is possible for the digital broadcast receiving apparatus 7100 to simultaneously receive two 
programs and to record the programs in the recorder 71 70, which is composed of a hard disk or another storage device. 
[0384] The controller 71 50 controls the channels of the programs that the receiving units 5110 and 71 10 receive, the 
location in the recorder 7170 of the data from the receiving units 5110 and 7110, the input into the replaying unit 5130 
10 or the recording unit 5120, and the like. Here, the input into the replaying unit 5130 or the recording unit 5120 means 
that the controller 71 50 selects one out of the receiving units 511 0 and 7110, and the recorder 71 70 when the controller 
71 50 gives the replaying unit 51 30 or the recording unit 5120 data from the receiving unit 5110, the receiving unit 711 0, 
or the recorder 7170. 

[0385] Under the control of the controller 7150, one of the receiving units 5110 and 7110 receives a program that 
15 has been designated by the user, and the other receives a recommendable program that has been determined ac- 
cording to the user information 6400. 

[0386] The recorder 71 70 is used as a ring-shaped recording buffer by the controller 71 50, and records the data that 

has been received by the receiving units 5110 and 7110 in the latest two hours. 

[0387] In the digital broadcast receiving apparatus 7100, the construction of which has been described, when the 
20 button image shown in Fig. 32 is pressed by the user after the start of a recommendable program that has been 
recommended according to the program recommendation function, the controller 7150 has the replaying unit 5130 
replay the recommendable program from the beginning using the data stored in the recorder 7170. For this purpose, 
the controller 7150 has one of the receiving units 5110 and 7110 receive a recommendable program and has the 
recorder 7170 record the recommendable program when displaying the button image shown in Fig. 32. 
25 [0388] Meanwhile, when a button image for recommending the user to record a presently watched program because 
the program ends after the usual bedtime (refer to Fig. 34) is displayed according to the bedtime responding function, 
and when the user presses the displayed button image, the controller 7150 has the recording unit 5120 record the 
program from the beginning using the data stored in the recorder 7170. As a result, it is possible to record a program 
from the beginning even when the recording is instructed after the program has started. 
30 [0389] While the user interface apparatus according to the present invention has been explained according to the 
embodiments, as a matter of course the present invention is not limited to the embodiments. Other examples of the 
present invention will be described below. 

[0390] (1 ) While the user is supposed to operate a recording and replaying apparatus or a digital broadcast receiving 
apparatus in the first to fourth embodiments, the present invention does not limit an apparatus operated by the user. 
35 [0391] More specifically, the user interface apparatus according to the present invention may be used in an analog 
broadcast receiving apparatus, a DVD player, a car navigation system, an electrical appliance, a home automation 
system, and the like. 

[0392] A user interface apparatus in a home automation system that has a recommend display appear according to 
user information as described in the fourth embodiment has a recommend display "fill a bath?" appear on the control 
40 panel. In this case, when the user instructs affirmatively, the bath is filled with hot water. 

[0393] (2) While a broadcasted program table is received in the first to fourth embodiments, a program table may be 
obtained through a network such as the Internet and may be recorded on and read from a storage medium such as a 
CD-ROM. 

[0394] (3) While an example in which an auxiliary menu item "preset for recording of next episode" is displayed on 
4$ the auxiliary menu after the preset for recording in the first embodiment, it is possible to display an auxiliary menu item 
"replay next episode" on the auxiliary menu after the replaying of the second episode and to display the auxiliary menu 
item "preset for recording of next episode" after the replaying of the fifth episode when five episodes have been re- 
corded. For this purpose, it is enough only to manage the information on the recorded serial such as the title and to 
display the auxiliary menu that fits for the situation after the judging whether the next episode has been recorded or 
50 next episode is to be broadcasted on referring to the Information and a broadcast schedule. 

[0395] (4) The user interface shown in the first embodiment, which automatically displays an auxiliary menu according 
to the apparatus condition and the past user operation, and the user interface in the second embodiment, which displays 
an auxiliary menu including keywords and function items in response to user operation may be provided in one appa- 
ratus. In addition, these two user interfaces may be provided in one apparatus as well as another user interface such 
55 as a user interface that displays an auxiliary menu with which a function that has been executed may be easily selected 
according to a user operation, in such a case, one of these user Interfaces may be selectively used. 
[0396] Meanwhile, while items displayed on the auxiliary menu are determined according to the auxiliary menu item 
display condition in auxiliary menu item information (refer to Fig. 8) in the first embodiment, it is possible to determine 
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the auxiliary menu items by calculating the evaluation values of the function items according to the semantic relation 
information 3530 and the function item/keyword condition Information 3540 shown in the second embodiment (refer 
to Fig. 23). More specifically, it is possible for the auxiliary menu Item determining unit 1416 to regard a piece of 
information in execution instruction information and apparatus condition information as an equivalent of a keyword in 

5 the second embodiment, to regard the value of a piece of information in execution instruction information and apparatus 
condition information as the input value into the node corresponding to a keyword, to update the evaluation values of 
the nodes corresponding to the function Items one by one according to the principle as shown in Fig. 23, and to de- 
termine function items to be displayed on the auxiliary menu according to the evaluation values heki by the nodes 
corresponding to the function items. 

10 [0397] Note that for the purpose to display an auxiliary menu with which a function that has been executed may be 
easily selected, it is enough to give a serial number to execute control information whenever the operation receiving 
unit 1411 creates execute control information, to store execute control information Including a given serial number in 
the execution instruction/apparatus condition information storage unit 1412. and to set the order of the function items 
to be displayed on an auxiliary menu on referring to the serial numbers of the execute control information corresponding 

'5 to the function items. In this case, it is also possible to display function item names that are written on button images 
In standard menus, which correspond to large items, middle items, and small Items In the execute control information, 
as the function items on an auxiliary menu. 

[0398] (5) The form of an auxiliary menu according to each of the first and second embodiments is not limited to that 
shown in each of the first and second embodiments. Any number of button images may be displayed in an auxiliary 
20 menu in any form. For Instance, the auxiliary menu item determining unit 1416 determines the auxiliary menu items 
according to the priorities in auxiliary menu item information in the first embodiment. It is possible to position button 
images according to the priorities of the corresponding functions. More specifically, it is possible to position a button 
image of the function with the highest priority at the part that is most accessible for the user. 

[0399] In addition, while a function to be selected by the user is displayed by a button image on which the name of 
2S the function Is written with a character string in the first to fourth embodiments, a function may be displayed in any form 
on the screen. For instance, a function may be represented by an icon of a self-explanatory image of the function. 
[0400] (6) While received by the remote controllers 1 300 and 5300 In the first to fourth embodiments, user operation 
may be received by a keyboard, a pointing device such as a mouse, and the like. 

[0401] While displayed with images on the screen in the first to fourth embodiments, the information that may be 
30 selected by the user (choices, and the like) may be expressed with voice, for instance. When keywords, i.e., choices 
are expressed with voice, the keywords are audibly expressed. When inputting the selection of a keyword, the user 
performs a voice input using an input unit such as a remote controller, a keyboard, and a mouse, or an audio recognition 
unit at the time the keyword is audibly expressed. 

[0402] In addition, while the remote controller 1 300 is equipped with two kinds of remote control buttons, i.e., remote 
35 control buttons for operating a standard menu and for operating an auxiliary menu, a remote controller may be equipped 
with one kind of remote control buttons so that a standard menu and an auxiliary menu woukJ be operated with the 
same remote control buttons and one of the menus would be operated at one time. In this case, however, the benefit 
of the remote controller 1300 may not be enjoyed. More specifically, a user interface with which a cursor Is positioned 
on each of two menus on the screen at the same time is not available without the remote controller 1 300. 
40 [0403] Note that when the controller 11 10 in the first embodiment displays an auxiliary menu after a standard menu 
has been displayed, it is possible not to disturb user operation of the standard menu by controlling the cursor so that 
the cursor would not positioned on the auxiliary menu at first even with a remote controller equipped with one kind of 
remote control buttons. 

[0404] (7) The "large item", "middle item", and "small Item" in the execute control information stored in the execution 
45 instruction/apparatus condition information storage unit 1412 in the first and second embodiments may or may not 
correspond to the hierarchical structure of the standard menus. 

[0405] The "setting 1", "setting 2", "setting 3", "object 1", and the like in the execution instruction Information, and 
the apparatus condition information stored in the execution Instruction/apparatus condition information storage unit 
1412 are not limited to the contents shown in the first and second embodiments. Each piece of information stored in 

so the execution instruction/apparatus condition information storage unit 1412 may have any content and may be used 
for the judgement of the condition for determining the auxiliary menu items on an auxiliary menu in the first embodiment. 
[0406] In addition, while execution management information in execute control Information is supposed to have the 
value representing "not decoded", "wait for execution", and the like, the value is not limited to these examples. The 
operations by the function execution controller 1413 in the first embodiment is not limited to the operations shown in 

ss the flowchart in Fig. 12. The operations may be anything when the execution of a functk)n Is controlled and the display 
timing of an auxiliary menu is given (steps S1 11 and S1 16) according to the execute control Information that reflects 
user operation. For instance, it is possible to suppose that execution management information has no value represent- 
ing "wait for execution" and to regard a preset for recording as the execution of one function to be treated equally to 
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an immediately-executed-type function. An auxiliary menu display timing may be given when a few seconds have 
elapsed since user operatbn was received. 

[0407] (8) While the value of information in execute control information and apparatus condition information stored 
in the execution instruction/apparatus condition information storage unit 1 41 2 is used as the condition of the judgement 

5 for determining the auxiliary menu item on an auxiliary menu in the first embodiment, the information related to user 
operation and/or the information related to the condition of the elements in the apparatus may be used as the condition. 
[0408] (9) The auxiliary menu display timing controller 1415 in the first embodiment may instruct the auxiliary menu 
item determining unit 1416 to determine an auxiliary menu item to be displayed on an auxiliary menu a plurality of 
times every time a predetermined period of time has elapsed since the timing given by the function execution controller 

10 1413. The auxiliary menu display timing controller 1 41 5 may adjust the timing independent from the the function exe- 
cution controller 141 3 and may instruct the auxiliary menu item determining unit 1416 to determine an auxiliary menu 
item to be displayed on an auxiliary menu. As a result, for instance, even when an auxiliary menu Is displayed during 
the display of a standard menu, the remote controller 1300 has two kinds of remote control buttons for operating a 
standard menu and an auxiliary menu, so that the user may operate both of the menus. 

IS [0409] (10) While the auxiliary menu controller 1418 deletes the auxiliary menu when 10 seconds has elapsed since 
the last operation of the auxiliary menu in the first embodiment, the period of time is not limited to 10 seconds. The 
auxiliary menu controller 1418 deletes the auxiliary menu when the user presses a button on the remote controller 
1300 apart from the auxiliary menu operation button 1360, for instance. 

[0410] The auxiliary menu in the second embodiment may be deleted by the auxiliary menu controller 3418 when 

20 10 seconds have elapsed since the last operation of the auxiliary menu. 

[0411] (11) While auxiliary menu item priorities are referred to when auxiliary menu items are determined in the first 
embodiment, it is not necessary to determine auxiliary menu items according to the auxiliary menu item priorities. 
[0412] Note that even when auxiliary menu items are determined according to the priorities, the priorities may be 
changed. For instance, the priority of an auxiliary menu item corresponding to a function that has been selected by the 

2S user may be increased. In this case, the more a user interface is used, the more useful the user interface. 

[041 3] (12) Even when a keyword or function item on the primary side has a semantic relation to that on the secondary 
side, the keyword or function item on the secondary side does not always have a semantic relation to that on the 
primary side in the semantic relation information 3530 in the second embodiment. It is possible that when a keyword 
or function item on the primary side has a semantic relation to that on the secondary side, the keyword or function item 

30 on the secondary side always has a semantic relation to that on the primary side. 

[041 4] In addition, while the weights are all "1 " in the second embodiment, a weight is not limited to "1 Mn this case, 
each of the outputs V[0], V[1], and the like in Fig. 23 Is weighted and input according to the weight. 
[041 5] (13) The display keyword determination by the auxiliary menu controller 341 8 may not be performed according 
to the processing shown in the flowchart in Fig, 24. For instance, the operations at steps S502 and S507 may be 

35 excluded from the processing. 

[0416] The auxiliary menu controller 3418 may not update keyword button images when the user selects a keyword 
and may update keyword button images only when the button image "display other words" is selected. In other words, 
the operation at step S403 after the operation at step S409 may be skipped and the operation at step S404 may be 
performed just after the operation at step S409 in the flowchart in Fig. 22. 

40 [0417] (14) While a button image is displayed on the monitor in the fourth embodiment (refer to Fig. 32), a plurality 
of button images may be displayed when a plurality of programs are recommended. In addition, it is possible to display 
a button image for displaying a recommendable program and another button innage for recording the recommendable 
program. In this case, when the user selects a button image, the apparatus performs according to the content of the 
selected button image. 

45 [041 8] (15) While the taste genre 6401 shows 1 0 or less user favorite program genres and the favorite program 6402 
shows 10 or less user favorite programs in the fourth embodiment, the number of the maximum number is not limited 
to " 1 0". In addition, priorities may be given to the user favorite program genres and user favorite programs. For instance, 
when priorities are given to the user favorite program genres, when the highest priority is given to sport, the second 
highest priority is given to news, and the third highest priority is given to movie, and when a plurality of recommendable 

50 progriams are detected according to the user favorite program genres, it is possible to select a predetermined number 
of programs in order of decreasing priority of genre and to display the selected programs as the recommend display 
[0419] (1 6) A button image for the recommend display that has been explained in the third and fourth embodiments 
may be dismissed by the user using the remote controller. The button image "return to movie "XX" (one minute has 
elapsed)" may be dismissed when the user switches channels to watch the movie "XX" with the remote controller. 

55 [0420] (17) The watching determination of one program is detected when the user has been watching a program on 
the same channel for at least 10 minutes in the third and fourth embodiments, the period of time is not limited to 10 
minutes. It is possible to detect the watching determination of one program by judging the same program has been 
watched for at least 10 minutes on referring to the program table. 
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[0421] ( 1 8) While a program that is to be broadcasted by the time a presently watched program ends is recommended 
according to the program recommendation function In the fourth embodiment, the programs that are to be broadcasted 
within a few minutes of the present time may be recommended every few minutes. 

[0422] (19) While the reason that is included in the content of the button image for a recommend display such as 
5 ("YZ" appearing)" is a genre or a player name according to the program recommendation function in the fourth em- 
bodiment, another reason such as the fact that the program has been watched or is watched every week may be 
included. In addition, the reason may be shown by a character string or a symbol that represents the reason. 
[0423] (20) While the user Information is not divided for a plurality of users in the fourth embodiment, the user infor- 
mation may be divided for a plurality of users. For instance, sets of a user name and the corresponding user information 
10 may be input, and a user name may be attached to a recommend display such as a recommendable program. In 
addition, each of the plurality of users may has a different remote controller. In this case, it may be judged that which 
remote controller Is now operating a digital broadcast receiving apparatus, and a recommend display related only to 
the user of the remote controller may appear according to the judgement. 

[0424] (21 ) While a recommend display appears after the watching determination when a presently watched program 
IS ends after the usual bedtime according to the bedtime responding function in the fourth embodiment, the timing of the 
recommend display is not limited to this example. The recommend display may appear when the bedtime draws near. 
In addition, the recommend display may appear when a program is scheduled to end at least 30 minutes after the 

bedtime. 

[0425] (22) It is possible to distribute a computer program that has the processing procedure of the execute control 
20 function and the user interface control function by the controller 1110 in the first embodiment (the operations in the 
flowcharts in Figs. 12 and 14, and the like), the processing procedure of the auxiliary menu control by the auxiliary 
menu controller 3418 in the second embodiment (the operations in the flowcharts Figs. 22 and 24, and the like), or the 
processing procedure of the recommend display control in the third or fourth embodiment (the operations In the flow- 
charts Figs. 27,. 31 , and 33, and the like) be executed by a general purpose computer or an electrical appliance equipped 
25 with a program execution function via a variety of communication channels, or to record the computer program on a 
recording medium and distribute the computer program. Such a recording medium Includes an IC card, an optical disk, 
a flexible disk, a read only memory (ROM), and the like. Generally speaking, when used, the distributed computer 
program is installed in an electrical appliance, a personal computer, and the like which each have a program execution 
function. Such an electrical appliance and a personal computer execute the computer program to realize the functions 
30 related to the user interface shown in the first to fourth embodiments. 

[0426] Although the present invention has been fully described by way of examples with reference to the accompa- 
nying drawings, it is to be noted that various changes and modifications will be apparent to those skilled In the art. 
Therefore, unless such changes and modifications depart from the scope of the present invention, they should by 
construed as being included therein. 

3S 

Claims 

1. A user interface apparatus that receives a user operation of an electrical device and notifies the electrical device 
40 of an execution instruction relating to an action according to the user operation, comprising: 

predicting means for predicting at least one action, an execution of which is supposed to be desired by a user; 
user interface means for providing a user interface in order to have the user designate one out of the actions 
that have been predicted by the predicting means, and for receiving a designation of an action by the user; and 
45 notifying means for notifying the electrical device of an execution instruction relating to an action according to 

the user action designation that has been received by the user interface means. 

2. The user interface apparatus according to Claim 1 , wherein the user interface means automatically provides the 
user interface without being instructed by the user. 

so 

3. The user interface apparatus according to Claim 2, further comprising user Infomnatlon storage means for storing 

user information that is peculiar to the user, wherein the predicting means predicts the actions according to the 
user information after a condition of the electrical device becomes a predetermined condition. 

55 4. The user Interface apparatus according to Claim 3, wherein 

the user information is taste information on tastes of the user, and 

the predicting means predicts actions that meet the tastes of the user according to the taste information. 
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5. The user Interface apparatus according to Claim 3, wherein 

the user Information is behavioral information on habitual behavior of the user, and 
the predicting means predicts actions that are necessary to keep the habitual behavior of the user according 
s to the behavioral information. 

6. The user interface apparatus according to Claim 3, wherein the user interface means provides the user interface 
by displaying at least one image in order to have the user designate one out of the actions that have been predicted 
by the predicting means, wherein the image displayed in the user interface includes an Image that shows a reason 

10 why the action has been predicted. 

7. The user interface apparatus according to Claim 3, wherein 

the electrical device is a broadcast receiving apparatus that displays an image of a received program on a 

monitor, 

the user information storage means stores user information that is peculiar to a user of the broadcast receiving 
apparatus, and 

the user interface means provides the user interface by displaying an image on the monitor for having the user 
designate one out of the actions that have been predicted by the predicting means. 

8. The user interface apparatus according to Claim 7, further comprising: 

program table storage means for storing a program table including information on a plurality of programs; and 
user information receiving means for receiving input of the user informatton by the user and for storing the 
user Information in the user information storage means, wherein 

the user information is program taste information that shows tastes of the user relating to programs, and 
the user interface means refers to the program table and the program taste information, detects a program 
that meets the tastes of the user other than a program that the broadcast receiving apparatus Is presently 
receiving, and displays an Image on the monitor for having the user designate an execution of an action of 
switching a received object of the broadcast receiving apparatus to the detected program. 

9. The user interface apparatus according to Claim 7, further comprising program table storage means for storing a 
program table including infomiation on a plurality of programs, wherein 

the user information is bedtime information that shows a bedtime of the user, and 

the user interface means refers to the program table and the bedtime information, compares the bedtime with 
a time when a program that the broadcast receiving apparatus is presently receiving ends, and displays an 
image on the monitor for having the user designate an execution of an action of recording the program when 
the program ends after the bedtime. 

10. The user interface apparatus according to Claim 2, further comprising: 

general operation receiving means for receiving a user operation of the electrical device other than the user 

action designation that the user interface means receives; 

general notifying means for notifying the electrical device of an execution instruction according to the user 
operation that has been received by the general operation receiving means; and 

holding means for holding execution instruction information on the execution instruction according to the user 
operation that has been received by the general operation receiving means, wherein 
the predicting means predicts actions according to the execution instruction information that is held by the 
holding means. 

11. The user interface apparatus according to Claim 10, wherein 

the predicting means predicts actions when an execution completion of an action of the electrical device is 
55 detected, and 

the user interface means provides the user Interface when the predicting means has predicted the actbns. 

12. The user interface apparatus according to Claim 11, wherein 
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the electrical device is a broadcast program recording and replaying apparatus that records a plurality of 
broadcast programs, replays a recorded broadcast program, and displays the replayed broadcast program on 
the monitor, 

when it is detected that one of a series of broadcast programs has been replayed by the broadcast program 
5 recording and replaying apparatus, the predicting means predicts an action of replaying a next broadcast 

program in the series of broadcast programs as one of the predicted actions, and 

the user interface means provides the user interface by displaying at least one image for having the user 
designate one out of the actions that have been predicted by the predicting means. 

10 13. The user interface apparatus according to Claim 11 , wherein 

the predicting means includes a priority storage unit for storing priority information in which priorities are set 
for actions of the electrical device. 

the predicting means predicts actions, a number of which Is at least one and no greater than a predetermined 

number, by referring to the priority information, 

the general operation receiving means has an operation button, which is used by the user, displays an image 
for assisting a user operation, and receives a user operation corresponding to the Image via the operation 
button, 

the user interface means has a designation button, which is different from the operation button and is used 
by the user, and receives the user action designation via the designation button, and 
when an execution of the action according to the user action designation that has been received by the user 
interface means needs setting Information according to a user operation, the notifying means creates an ex- 
ecution instruction that includes the setting information by referring to the execution instruction infornnation 
that is held by the holding means, and notifies the electrical device of the created execution instructiwi. 

14. The user Interface apparatus according to Claim 10, wherein the predicting means predicts actions when comple- 
tions of all user operations that are necessary to have the electrical device execute one action are detected. 

15. The user interface apparatus according to Claim 10, wherein when an execution of the action according to the 
user action designation that has been received by the user interface means needs setting information according 
to a user operation, the notifying means creates an execution instruction that includes the setting information by 
referring to the execution instruction Information that is held by the holding means, and notifies the electrical device 
of the created execution instruction. 

35 16. The user interface apparatus according to Claim 10, wherein 

the predicting means includes a priority storage unit for storing priority information in which a priority is set for 

an action of the electrical device, and 

the predicting means predicts actions, a number of which is at least one and no greater than a predetermined 
40 number, by referring to the priority information. 

17. The user interface apparatus according to Claim 2, further comprising general operation receiving means for re- 
ceiving a user operation of the electrical device other than the user action designation that the user interface 
receives, wherein 

the general operation receiving means has an operation button, which is used by the user, displays an image 
for assisting a user operation, and receives a user operation corresponding to the image via the operation 

button, and 

the user interface means has a designation button, which is different from the operation button and is used 
by the user, and receives the user action designation via the designation button. 

18. The user interface apparatus according to Claim 2, wherein 

the electrical device is a broadcast receiving apparatus that displays an image of a received program on a 
55 monitor, 

the electrical device includes: 

receiving means for receiving a program on a channel; and 

condition detection means for detecting a condition where the receiving means had received a program on a 
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first channel for at least a first period, the first channel was switched to a second channel according to a user 
operation, and a second period has elapsed, 

the predicting means predicts a switching action to the first channel as one action, an execution of which Is 

supposed to be desired by the user when the condition Is detected by the condition detection means, 
the user interface means displays an image on a monitor for having the user designate an execution of the 
switching action to the first channel that has been predicted by the predicting means, and receives the user 
action designation, and 

when notified of the switching action to the first channel by the notifying means, the broadcast receiving ap- 
paratus controls the receiving means and has the receiving means receive the program on the first channel. 

19. The user interface apparatus according to Claim 1 , further comprising: 

evaluation value storage means for storing evaluation values that each correspond to different actions of the 

electrical device; 

semantic relation Information storage means for storing semantic relation information on semantic relations 
between the actions of the electrical device and a plurality of keywords; 
keyword receiving means for receiving a designation of a keyword by the user; 

Initializing means for initializing the evaluation values that are stored In the evaluation value storage means 
and setting each of the evaluation values at a predetermined value; and 

evaluation value increasing means for referring to the semantic relation information and for increasing an 
evaluation value corresponding to an action that has a semantic relation to the keyword, the designation of 
which has been received by the keyword receiving means, wherein 

when the keyword receiving means receives the keyword designation, the predicting means predicts actions 
by specifying a predetermined number of actions in decreasing order of evaluation value according to the 
evaluation values that are stored in the evaluation value storage means, and 

the user interface means provides a user interface for having the user designate one action out of the prede- 
termined number of actions, which have been predicted by the predk^ting means. 

20. The user interface apparatus according to Claim 19. wherein 

the keyword receiving means receives the designation of the keyword after creating a dedicated menu for 
having the user designate an action of the electrbal device that includes a plurality of keywords having semantic 
relations to at least one of the actions of the electrical device by referring to the semantic relation information 

and displaying the dedicated menu, and 

the user interface means provides the user interface by displaying an image for having the user designate 
one out of the predetermined number of actions. 

21. The user interface apparatus according to Claim 20, further comprising: 

general operation receiving means for receiving a user operation of the electrical device other than the keyword 
designation that the keyword receiving means receives and the user action designation that the user interface 
means receives; and 

general notifying means for notifying the electrical device of an execution instruction according to the user 

operation that has been received by the general operation receiving means, wherein 

the general operation receiving means has an operation button, which Is used by the user, displays an Image 

for assisting a user operation, and receives a user operation corresponding to the image via the operatbn 

button, 

the user Interface means has a designation button, which is different from the operation button and is used 

by the user, 

the keyword receiving means receives the user keyword designatbn by the user via the designation button, and 
the user Interface means receives the user action designation via the designation button. 

22. A broadcast receiving apparatus that displays an image of a received program on a monitor, comprising: 

execute control means for receiving an instruction, for controlling elements in the broadcast receiving appa- 
ratus In response to the instruction to have an action be executed; 

user information storage means for storing user information that is peculiar to a user of the broadcast receiving 

apparatus; 
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user interface means for displaying at least one image for having the user designate an action out of at least 
one action, an execution of which is supposed to be desired by the user according to the user information that 
is stored in the user information storage means, and for receiving a designation of the action by the user after 
the broadcast receiving apparatus Is in a predetermined condition; and 

notifying means for notifying the execute control means of an execution instruction relating to an action ac- 
cording to the user action designation that has been received by the user interface means. 

23. The broadcast receiving apparatus according to Claim 22, further comprising program table storage means for 
storing a program table including information on a plurality of programs, wherein 

the user information is program taste information that shows tastes of the user relating to programs, and 
the user interface means refers to the program table and the program taste information, detects a program 
that meets the tastes of the user other than a program that the broadcast receiving apparatus is receiving, 
and displays an image on the monitor for having the user designate an execution of an action of switching a 
received object of the broadcast receiving apparatus to the detected program. 

24. The broadcast receiving apparatus according to Claim 23, further comprising user information receiving means 
for receiving the user information input by the user and for storing the user information in the user information 
storage means. 

25. The broadcast receiving apparatus according to Claim 23, further comprising received program information ob- 
taining means for obtaining information on the received program by referring to the program table, and for storing 
the obtained information in the user information storage means as the program taste informatton. 

26. The broadcast receiving apparatus according to Claim 22, further comprising program table storage means for 
storing a program table including information on a plurality of programs, wherein 

the user information is bedtime information that shows a bedtime of the user, and 

the user interface means refers to the program table and the bedtime information, compares the bedtime with 
a time when a presently received program ends, and displays an image on the monitor for having the user 
designate an execution of an action of recording the presently received program when the program ends after 
the bedtime. 

27. The broadcast receiving apparatus according to Claim 26, further comprising temporary storage means for tem- 
porarily storing data on the received program, wherein when notified of an execution instruction for the action of 
recording the received program by the notifying means, the execute control means has recording means in the 
broadcast receiving apparatus record the received program from a part that has been received before a notification 
of the execute instruction for the action of recording using the data stored in the temporary storage means. 

28. The broadcast receiving apparatus according to Claim 22. wherein an image that the user interface means displays 
includes an image that shows a reason why an action is predicted. 

29. A broadcast receiving apparatus that displays a received image on a monitor, comprising: 

receiving means for receiving a program on a channel; 

condition detection means for detecting a condition in which the receiving means had received a program on 
a first channel for at least a first period, the first channel was switched according to a user operation, and a 

second period has elapsed; 

user interface means for displaying an image on the monitor in order to have a user designate an execution 
of a switching action to the first channel when the condition detection means detects the condition, and for 
receiving a designation by the user; and 

reception control means for controlling the receiving means and for having the receiving means receive the 
program on the first channel when the user interface means receives the designation by the user. 

30. The broadcast receiving apparatus according to Claim 29, wherein the image that the user interface means displays 
includes an image showing that the second period has elapsed since the first channel was switched. 

31. A storage medium that records a program for having a computer execute a user interface process for receiving a 
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user operation of an electrical device and for notifying the electrical device of an execution instruction of an action 
according to the user operation, the user interface process, comprising: 

a prediction step for predicting at (east one action, an execution of which is supposed to be desired by a user; 
s a user interface step for providing a user interface in order to have the user designate one out of the actions 

that have been predicted at the prediction step and for receiving a designation of an action by the user, and 
a notification step for notifying the electrical device of an execution Instruction relating to an action according 
to the user action designation that has been received at the user Interface step. 

10 32. A storage medium that records a program for having a computer execute a broadcast reception process for dis- 
playing an image of a received program on a monitor, the broadcast reception process, comprising: 

a user Interface step for displaying an Image in order to have a user designate an action out of at least one 

action, an execution of which is supposed to be desired by the user according to user information that is 
'5 peculiar to the user after the computer is in a predetermined condition, and for receiving a designation of an 

action by the user; and 

an execution step for executing an action according to the user action designation that has been received at 
the user interface step. 

20 33. A storage medium that records a program for having a computer execute a broadcast reception process for dis- 
playing a received image of a received program on a monitor, the broadcast reception process, comprising: 

a reception step for receiving a program on a channel; 

a condition detection step for detecting a condition in which a program on a first channel had been received 
25 for at least a first period, the first channel was switched according to a user operation, and a second period 

has elapsed; 

a user interface step for displaying an Image on the monitor In order to have a user designate an execution 
of a switching action to the first channel when the condition has been detected at the condition detection step, 
and for receiving a designation by the user; and 
30 a reception switch step for switching a received channel so that the program on the first channel is received 

when the designation by the user has been received at the user interface step. 



35 



40 



45 



50 



55 



37 



'4 H\ « 

EP0 940 980A2 



CO 



C\3 CO 



o 
o 

CV] 
CD 



\ 



o 



o 
o 

05 



\ 







o 


















00 




o 

So 



8<^ 



o 
o 

CO 



7 




O o o 
CS3 CO 

52 CO CO 
cn 



38 



EP 0 940 980 A2 



FIG. 2 



REPLAY- 



PRESET FOR- 
RECORDING 



RECORDING 
^ TV 

SETTING — 



^ NORMAL SPEED 

■> DOUBLE SPEED 

^ DETECT AND REPLAY 



^NORMAL PRESET 
GCODE 



39 



EP 0 940 980 A2 




CO 

d 

IX, 



40 



EP0940980 A2 




42 



EP0940980 A2 



CD 



2 
O 

H 
O 

(33 

3 Q 

ac c 

c o <: 

H o 
H W V 

QQO 



< 
-J 

CL, 

Q 
2 
< 

H 
O 

H 

W 

Q 



< 



Q 
U 
W 

Oh 

00 

►J 

OQ 

o o 

2 Q 



Q 

Oh 
Oh 

00 



< 



< 

Oh 

(JLl 



o 
o 

Oh 



OQ 



2 
O 

o 

w 

H 
Q 

- w Q 

H 



W O 

JO 



DC 



00 



2 

O 

> IX, 

OO w 
W W ^ 
H H 2 

CJLl U 

QQO 



[3 

Oh 



O 
Q 



2 
O 

H 
< 

2 2 
O O 

I— I 

Q 
2 
O 

o 

H 
CiJ 

CO 

W 
Dei 

Oh 



Q 

H 
O 

§ 

Q 

H 



5 2 



13 

00 



2 



ctj o 



2 
O 

W 



o 

H 

00 

w 

Oh 



^ ^ ^ ^ ^ ^ ^ 



02 

wo 



o 

2 

l-H 

Q 

o 
o 



i 



1 



w 

(—1 

p 
< 

Ou 
W 

Pu 
O 

2 

O . 

12 • 

w 

Q 



2 
O 

w 

Q 
U 



O 
2 



00 



t ^ \ 



i 



i 



43 



EP 0 940 980 A2 



CNJ 



\ 



o 

H 
Q 

O 

o 

00 

D 
H 



So 



2 
O 

H 
< 

o 

2 

2 
O 

E- 
O 
D 
Dei 
H 
00 

2 

2 
O 

D 
O 

w 

X 



t3 

CO - 

a 



.CO 



<CQ 



3 
S 

CO 



PQ 
O 



CD 



CO 



CD 



CD 




(j-i 

OS 



^^^^ 



T 

o 



z 



2 

o 



Q 
2 



is 



CQ 

O 
o 



CiJCiJ 
CQ< 
JOO 

<Q 



00 

Q 



44 



EP 0 940 980 A2 




45 



EP 0 940 980 A2 



w 








1— 1 


2 

O 


Q 


[ATI 












o 








2 




o 


2 




O 


O 


H 


= DELETI 


INSTRUC 


II 


w 


2 




o '. 


< 






Di 


ITEM 


EXEC 


D 


D i 


2 


2 ' 


W 


WL 














< 


< • 


►J 




X 




D 




< 


< - 



-<CPQ 



t3 



"3 



1 



■i 



•COS 



CP 




00 00 



II 

H 

o 

Dl, 

H 

D 
2 



< 



to 



< 



00 



00 



46 



EP0940 980 A2 



o 
o 

Q 

O 
O 
W 

a: 
o 

w 

II 

w 

< 

D 

DtiJ 
<2 



2 
O 

o 
w 

X 
w 

D 

>^ 

< 
J 

< 



CZ> CUD 



u 



CO 



CO 



CD 



• 00 = 



o 



SMALL 

ITEM 
II 


DETEa 
FROM 
PROGRAM 
TABLE 


MIDDLE 

ITEM 
11 


DETECT AND 
PRESET 




LARGE 

ITEM 
II 


PRESETFOR 
RECORDING 





Q 
< 

O 



-\ ^ (rt •4—1 

^ o) 0) a> 
OO dO S) DO 



s 

< 
< 

2 
O 

O 



< 

CL, 



II 



Q 
W 
f- 
D 
O 
W 

0^ II 



CVJ CM 
OO OO 



47 



i 



EP 0 940 980 A2 



FIG. 11 
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FIG. 13 
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FIG. 14 
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FIG. 15 
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FIG. 20 





FUNCTION ITEM NAME 1 
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10 
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11 
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12 
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13 
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